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ABSTRACT 
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indicators development are presented in this document. Research was 
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Initiative," by Peter A. Easton, Dwight R. Holmes, C. Howard 
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Concept of Educational Efficiency: Eight Years of lEES Project 
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Educational Management Information System: The Case of Botswana," by 
Shirley A. Burchfield; "The Development of an Educational Information 
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Identification of Indicators of Efficiency and Quality to Inform 
Curriculum Development and Implementation Policy," by Kent L. Noel; 
and "The Impact of Sectoral Adjustment on the Design and 
Implementation of an Educational Management System: The Case of 
Guinea," by Joshua Muskin. (LMl) 
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PREFACE 



Over the last decade, educational development programs increasingly have included 
information systems as an integral component. It has become increasingly espoused, if not 
accepted that effective educational planning and development is predicted on collecting and 
using representative, quality data. The focus by many education and development agencies 
on Education Mans^ement Information Systems (EMIS's) and educational indicators 
underscores this fact. 

Though selective and abstract by their nature, EMIS's and indicators are an attempt to bring 
more complete data-based observations of the educational enterprise, or certain of its aspects, 
to policy makers, program managers, and other stakeholders. An EMIS is expected to 
provide regular, timely information on project or program status and implementation for 
management monitoring, decisions and actions. This type of system requires a fa' 1y deep 
level of trained personnel and technology, and a commitment to assign those resources to the 
task of developing and maintaining the EMIS. Indicator sets can provide a more 
comprehensive picture of the system, including trends, problem areas, and opportunities. 
Indicator sets also can highlight the need for action research or case studies, to fill qualitative 
gaps in national data systems. 

The collection of papers in this volume represents experience in EMIS and indicators 
development, conducted under the Improving the Efficiency of Educational Systems (lEES) 
Project. lEES Project assistance in information systems development has been undertaken in 
several countries, notably Indonesia, Botswana, Haiti, Guinea and Nepal. The information 
systems which are being developed in these countries vary in their context, purpose, scope, 
methodology, and selection of indicators and data. Consequently, the cases presented in this 
book offer a range of experiences, more than implementation of a single model in multiple 
sites. 

The first paper, by Peter Easton, D wight Holmes, Howard Williams, and Joy duPlessis, sets 
the theoretical background and framework for lEES Project work in the area of efficiency 
indicators. The development of the prototype education system model which is referred to in 
the case studies m this volume is described m this paper. A brief review of the literature in 
educational indicators in both the international and U.S. domestic arenas is included, along 
with conceptual exploration of key terms such as educational efficiency, educational quality, 
and indicators. Potential uses of ''indicator systems, along with several caveats, are 
outlined, and a series of follow-up studies is proposed. 
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In the second pa^r, the evolution of the "efficiency conc^t/ as operationalized by the EEES 
Project, is presented by Dwight Holmes. The work describ^ in the first paper is placed in 
historical context within the lEES Project along with other, parallel activities which have 
been undertaken. The elaboration and promotion of **efficiency** has been fundamental to 
major lEES initiatives, such as the extensive use of Sector Assessments, academic work, 
particularly by Douglas Wmdham and David Chapman, and lEES activities such as policy 
analysis training, and development of EMIS' and national education indicator sets. 

Holmes' description of lEES' efficiency-oriented project activities appears to indicate an 
evolutionary program along very definite Imes. As Holmes points out, however, this 
evolution has been largely dependent on country contexts, the initiative of participating 
individuals, and emergent opportunities. The theme of efficiency runs throughout. Yet 
greater care should be given to coherency and mtegrity of the activities, not to ensure the 
sanctity of efficiency, but to build upon these experiences. Holmes also highlights the need 
for an ancillary theme for efficiency initiatives: That, if the efficiency concept is to provide 
useful guidance for data*based policy and program decisions, then those decisions need to be 
continually linked to improvements at the level of educational delivery. 

In the case of Botswana, computer-based EMIS's have been developed in several Ministry of 
Education departments, including Planning, Bursaries, Teaching Service, Primary, 
Secondary, and Non-formal Education. Shirley Burchfield describes the need for 
coordination of these databases, and the strategy used for unifying the constituent data-bases 
into a system-wide data-base, allowing for overall system monitoring and efficiency analysis. 

In consultation with the departmental data-base managers and senior education officials, it 
was agreed that a model of educational efficiency indicators would best suit the need for a 
system-wide data-base. Burchfield describes the processes of indicator selection, data 
collection and analysis, and the technical,methodological, and organizational issues involved. 
Burchfield' s presentation also includes a listing of the efficiency indicators, and the system 
computer screens, illustrating how this system has been computerized. 

The paper by Howard Williams is a reflective description of educational information system 
development in Nepal. The educational information system is comprised of a Statistical 
Information System (SIS), action research, educational indicators, and evaluation standards. 
In developing this system, an organization development (OD) approach has been used, in 
order to assess what educational information and systems would be of most value and use to 
the Ministry of Education, under current circumstances. 

Each of the components of the information system is recognizable by its own conceptual 
base, methodology, and techniques. Yet no single component is a wholly complete, stand- 
alone operation, aside from the basic SIS already established by the alone operation, aside 
from the basic SIS already established by the Ministry. Instead, the consultative, interactive 
relationship used in the OD approach has produced a configuration of information functions 
and activities which appear to be most valuable and useful. 
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In the next ps^r, Kent Noel presents an zdaption of an indicator model for use in 
curriculum development and institutionaUzation in Botswana. Noel argues that curriculum 
development should be considered as a critical subsystem, with the attendant considerations 
normally given to the sector as a whole, such as: power sharing, communication and 
consultation, lines of authority for decision-making, production, and delivery, management 
and supervision. Consequendy, subsystems, such as curriculum development, warrant full 
performance information systems, such as the EMIS and indicator systems now being 
developed for the national education system. 

Based on six-years of experience with national curriculum development, Noel proposes a 
schematic outline of policy and management information issues to be addressed, including 
information clientele, ENQS components, data requirements, and suggested indicators. 
Noel's model follows the framework us^ for the national system in Botswana, but 
necessarily reflects the particular characteristics of curriculum development. 

Finally, in the case of Guinea, presented by Josh Muskin, system indicators and an EMIS are 
being developed in conjunction with **conditionalities*' for sectoral assistance. The 
Government of Guinea, and the participating donor agencies. The World Bank, United States 
Agency for International Development (USAID), and the French Government's Fond d'Aide 
at de Cooperation (FAC) have developed an educational sector reform program to improve 
quality, access and management. The **conditionalities*' serve as interim implementation 
targets, which must be met for continued funding of the reform. 

Muskin expresses his concern that the conditionalities, and their system indicators, could 
become ultimate goals for organizational performance, rather than serving as guideposts for 
the vision and achievement of reform, which is to deliver a higher quality education to more 
Guinean children. To mitigate against the organizational tendency of servicing the indicators 
instead of system clients, Muskin proposes a consultative approach to developing an EMIS, 
to improve the chances that the EMIS will better represent reform goals of service delivery, 
rather than being a reporting system for periodic funding. 

Earlier drafts of each of these ps^rs were presented at annual meetings of the Comparative 
& International Education Society: The pdptr by Easton et al., was presented at the 1991 
meeting at the University of Pittsburgh, while the remaining five papers were presented as 
part of the panel on "The Development and Use of Educational Data for Making Policy 
Decisions to Improve the Quality and Efficiency of Education" at the 1992 meeting, held in 
Annapolis, Maryland. The authors wish to thank Richard Pelczar, lEES contract manager, 
USAID/Washington, for his encouragmg and critical support for these various efforts. 
Thanks are also extraded to Simon Ju, lEES/Indonesia, for the guidance of his extensive 
work in information systems, and for his review of these papers. 



Dwight Holmes Howard Williams 

Tallahassee, Florida Kathmandu, Nepal 
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Collaboratiye Design of Educational Indicator Systems 
in Developing Countries: 
An Interim Report on an lEES Project Initiative 



Peter A. Easton, Dwight R. Holmes, C. Howard Williams & Joy duPlessis 



INTRODUCTION 

The design of educational indicator systems is presently a "hot topic** in both the industrial and 
developing worlds (Bottani & Delfau, 1990; Chapman, 1990; Odden, 1990; Windham, 1988a, 
1988b) as better use of existing data at all levels holds out the hope of rationalizing the 
macro-management of educational systems and directing attention to the real impediments to 
improved performance and greater equity. 

lEES Project activities in this domain were bom of some related concerns, though ones that 
changed form significantiy in the course of implementation. Evaluation of the first five years 
of lEES Project activity suggested a number of positive effects, but also a need to better 
document project impacts and the evolution of the baseline situation and socio-economic 
environment of the educational systems in question. Project staff therefore decided to begin 
work on a prototype indicator system for tracking the overall state of the education systems in 
participating countries, in part as an ongoing and greatiy simplified form of sector assessment. 

Once the team responsible for this effort began to bring together current research and thinking 
on indicator systems with its own experience in educational data use in developing countries, it 
quickly became clear that an additional two dimensions would be needed: First, incorporation 
into the model of qualitative data on educational pr: cesses, habitually the weak point of indicator 
systems; and, second, major and structural provision for host country counterpart participation 
in indicator system design and data definition, as well as anticipation of considerable variance 
and country specificity in design characteristics. 

In the rest of this paper, we briefly describe our conceptual analysis of educational indicator 
systems, the prototype model of primary education developed, and the process envisaged (and 
now underway) for host country elaboration and modification of the model. We will also discuss 
tile potential for utilizing tiie indicator system botii for in-country educational planning and staff 
training purposes. 
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n. CONCEPTUAL ANALYSIS OF EDUCATIONAL INDICATOR SYSTEMS 



A. The Nature of Indicators and Indicator Systems 

Recently there has appeared an increasing amount of literature about indicators for educational 
systems, provoked by concern with educational accountability in this country and spearheaded 
by the OECD Indicators Project in the rest of the industrialized world, as well as by donors' 
attempts to improve educational management and soften the blow of diminished educational 
resources in less-developed countries (LDCs). 

There is no uniform definition of the notion of an •'indicator/ however • Chapman (1990, p. 
229) terms indicators "proxies used to represent the underlying reality of a system or program** 
and notes that they are necessarily an •'oversimplification'' of this reality, Cobbe (1989, p, 3) 
makes a distinction in his monograph between raw data, statistics and indicators, noting that 
the latter should permit immediate - or nearly immediate - inferences about the performance 
of the [educational] system from the point of view of the objectives of the system/ 

Oakes (1986) de&ies an educational indicator as "a statistic about the educational system that 
reveals something about its performance or health"; and Smith (1988) further suggests' that it 
"either assesses or is related to a desired outcome of the education system or describes a core 
feature of that system." Odden (1990, p- 24) points out that an "indicator system" is 
considerably more than a single indicator or statistic. It should provide measures of various 
components of the educational system as well as information about how those components work 
together to produce the condition of the system and changes in the condition of the system over 
time. 

Most researchers identify indicators with composite indices of the operations of the educational 
system designed to reveal some critical aspect of its operation. Johnstone (1984) goes so far as 
to restrict the term to rather complex compounds of data providing a highly synthetic picture of 
the system. 

Such an approach seems unnecessarily restrictive, however. The essential notion is simply that 
of particularly meaningful data that point to and describe a significant characteristic of the 
educational system. In some cases, a single type of data may do this (Nuttall [1990] calls these 
••primary mdicators"); in other cases, more processed composites of raw data or statistics may 
be used ("secondary indicators," in Nuttall's parlance). In all cases, commentators tend to 
emphasize the fact that, as Bottani (1990, p. 337) puts it, "indicators do not explain; they only 
point." Some additional resources must be calloi upon in order to interpret the data that an 
indicator system produces. 

B. What is Worth Indicating? 

If indicator systems only point, what are we pointing at? From a generic and "rational" point 
of view, the underlying concern in indicator system development seems to lie with discovering 



whether and how - or "to what degree" and "in what way" - the educational system is 
accomplishing its goals. Since those goals will generally be quite country-specific, and given 
that there is often considerable debate within a country as to what those goals ought to be, it 
follows that indicator systems may also need to be quite idiosyncratic and subject to 
modification, a tendency tempered only by some measure of concern with consistency and 
comparison over space and time. 

At the same time, it is evident tliat the lEES Project, like other multinational endeavors of donor 
agencies, is predicated on the notion that there are some universal criteria of educational system 
performance, related in particular to issues of efficiency and quality. How universal are these 
criteria and how much uniformity can or should they lend to a multinational indicators project? 

Educational Efficiency: Efficiency in the delivery of educational services is a central concern 
of the lEES Project. It is the Project's "middle name." A good deal of conceptual work has 
been done on the topic under the aegis of the project, particularly the substantial monographs 
by Windham (1988a, 1988b). Much of that analysis is implicitly adopted here. At the same 
time, we feel constrained to note that injunctions to maximize the ratio of educational outputs 
or outcomes to educational inputs frequently glaze over three fundamental issues: 

First, technically speaking, the concept of efficiency is only meaningful if you correctly 
specify (and measure) the outputs or outcomes in question. Desired educational outputs and 
outcomes are not simple, and they may vary significantly from one country or region to 
another. They may involve affective as well as cognitive results, group as well as individual 
effects, and distributional as well as summational considerations. Policy makers may in reality 
be trying to maximize an entire vector or set of outcomes - Windham refers to it as the outcome 
"mix", - including (conceivably) the desideratum that the relative position of one group 
improve relative to that of another. Issues lite these are often circumvented by imposing at the 
outset a conventional series of objectives and desired outcomes — generally those in vogue with 
major donors. 

Second, efffciency is a ratio, not an absolute magnitude. Applying the perfect competition 
and general equilibrium postulates of neoclassical economics, it is generally assumed that 
efficient methods that are applicable at one resource level are generalizable across others as 
well. This may not be true; and, if not, the reasoning is in serious trouble. Also, for related 
reasons, as Cc^be (1989) points out, an efficiency criterion can be substantively ambiguous, 
particularly if a short-term view is adopted. For example, cutting inputs by one-half might 
"mathematically" improve efficiency in short-run perspective, yet have ruinous longer-term 
effects. 

Third, educational systems need to be efficient both in generating AND in expending 
resources, and these two objectives are not always perfectly compatible. A system which 
produces less ou^uts per unit of input but manages to generate or elicit more input - e.g., 
community or private financing - may be better adapted to accomplishing national objectives 
than a more "efficient" one. In a sense, this point mirrors the economic distinction between 
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technical and allocative efficiency, since the latter takes account of the relative prices of inputs 
and outputs and therefore the valuation that public or private) consume place on educational 
system performance. 

Overall, Windham suggests that "the efficiency concept is a neutral device" (1988, p. 9), but 
that only holds under one of two very restrictive conditions: (1) if the term is given no content, 
in which case it is operationally useless; or (2) if there is substantial agreement among all 
stakeholders concerning the objectives or the "ou^uts*" that are being maximized. To make the 
concept operational, some set of objectives or desired outputs must be specified; and the 
question then immediately becomes "Whose objectives? De^red by whom?" 

Educational Quality: Fuller (1986) chronicles four different phases in the definition of 
educational "quality" - from an initial stress on the mix of material outputs, through a period 
of increased emphasis on the importance of individual abilities and perceptions, to a phase of 
growing awareness of the centrality of cultural and social patterns that shape school process, and 
on to more recent awareness of the non-school-related and political objectives that "educational 
quality" campaigns may fulfill. Cobbe (1988, p. 3) identifies educational quality with the "value 
added" by the system, though this makes it nearly synonymous with the efficiency concepts 
discussed above. 

Easton and Cayhana's paper on the efficiency and quality of technical education in Indonesia 
(Easton & Cayhana, 1989, pp. 3-4) notes two main tendencies in efforts to operationalize the 
notion of educational quality: 

On the one hand, quality is defined as embodiment or approximation of characteristics that are 
socially accepted as proof of excellence. Thus, if all teachers in an academic secondary school 
have Master's degrees, the group will be considered a high quality staff. One the other hand, 
quality is defined as proven ability to produce results - in short, the argument that *a tree is 
known by its fruits.' 

Moreover, "educational quality** in the former sense is very often invoked in evaluations and 
policy studies for two reasons worth noting: First, as a substitute or proxy for efficiency 
measures, because genuine output data are so frequently unavailable; and second, because this 
open-ended concept is felt to capture more of the multiple and sometimes unquantifiable 
dimensions of educational outcomes than stricter input-output measures. Concern with 
educational quality therefore opens the door to the idea that other than quantitative data may 
need to be incorporated into an indicator system - or used in tandem with it - in order 
adequately to describe and analyze the current state of the educational system. 

C. Specifying the Underlying Conceptual Model 

Two points are made repeatedly in current literature on educational indicators: First, that 
indicators "do not explain, they simply point"; and second - partly as a consequence - that one 
needs to take some care to think out Uie underlying model of educational process on which the 
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indicators are based, and which will serve as a basis for interpreting them. The model need not 
be very complex. The Rand Corporation scheme presented in Odden (1990) includes only three 
types of inputs, four types of processes and three categories of outputs. But most authors 
maintain that some such model is implicit in any set of indicators anyway, and that it is therefore 
best to get the subject out in the open and examine it. 

We concur with the need to give some careful thought to the model of educational processes that 
underlies an indicator system. Most IRES studies and documents employ some form of a 
"CIPP- framework (in fiact -CIPOO- insofar as the -product- is differentiated into -output- and 
-outcome-). The CIPP framework is not really a model in the sense meant above, however, 
but simply a set of categories (based albeit on a production function analogy) for thinking about 
educational processes. The question remains, -What kinds of *inputs,* ^processes,' ^outputs' and 
^outcomes* will be put in the model?- and -Who will make the decision?" 

D. Choosing Indicators: Criteria and Methods 

The next step is choosing the specific indicators to be tracked. We feel that the choice should 
be made in accordance with both conceptual and practical criteria. Before discussing those 
criteria, however, there is a prior question concerning the -population- of possible indicators 
from which the choice will be made. In order to avoid overly or artificially constraining that 
initial set (and therefore all the choices made from it), we decided to inventory a large number 
of potential indicators, cross- classifying them by conceptual category or cluster. The choice 
criteria were then to be applied to this stratified grouping. 

Chapman (1990, p. 229) suggests that -the appropriateness of particular indicators is judged in 
terms of their fidelity with the underlying reality they rq)resent, the extent to which the 
indicators are relevant and understandable by the data users, and the extent to which data on 
these indicators can be collected and analyzed in a cost-effective manner.- We attempt to 
include these criteria and perspectives in the discussion below. 

1. Conceptual Criteria 

Conceptual criteria essentially involve considerations similar to the ones gov^ing the 
articulation of the underlying model. Since only a relatively few indicators will be 
systematically monitored, one wants to be sure that they relate to "key- aspects of the 
wiucaticMial system. What does -key- mean, however, and who decides which indicators meet 
this criterion? 

Considering the first question first - we think that the word -key- essentially means that the 
aspects of the educational system in question are judged (by appropriate parties and in an 
appropriate manner) to be particularly expressive of its current state or level of quality, or 
particularly critical to its performance in achieving (their) priority objectives. Two technical 
issues are at stake here: The first concerns validity, and the second has to do with causality. 
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The validity question involves the degree to which the particular indicators chosen validly 
represent the underlying characteristic of the educational system that one wishes to measure, an 
argument that can be made on either conceptual (construct validity) or empirical (concunent 
validity) grounds. The causality issue concems the degree to which these underlying 
characteristics are in fact related in some cause-and-effect manner to achievement of the given 
system performance objectives. 

On both grounds, we can use — though advisedly - some of the literature reviewed by FuUer 
(1986, 1987) and otiiers that seeks to identify key factors in improving educational quality in 
LDC schools. Obviously research on die countries in question has die greatest fzce validity, but 
is frequenUy hard to come by. 

2. Practical Criteria 

Practical criteria to use in choosing indicators are essentially of two kinds: (1) data availability 
considerations, and (2) data quality considerations. They will be broached in that order. 

Data availability considerations are relatively straightforward, but are nonetheless wortii careful 
examination. From the point of view of a centrally-administered international project, one could 
establish a hierarchy from ^'more to less available" with gradations something like the following: 

(a) Available in easily accessed international publications. 

(b) Available in more remote international publications. 

(c) Available in easily-accessed international data bases. 

(d) Available in national publications easily accessed in-country. 

(e) Available in national publications requiring specially-authorized access in-country. 
(0 Available in international data bases requiring special search or access. 

(g) Available in national data bases requiring special search or access in-country. 

(h) Available in-country in dispersed or raw data form requiring considerable 
collection or collation effort. 



The order and relevance of this scale is obviously different if one is situated at the national or 
regional level, and appropriate changes should be made. In any case, there is some implicit rank 
order of availability characterizing the different forms of data that one might want to include in 
an indicator system. 

Note tiiat the time period must also be specified in determining availability. Data for a given 
indicator may be fairly accessible for the 1988-90 time period, for example, but much more 
difficult to obtain for die 1984-87 period. 

Data quality: This is botfi tfie most critical and the touchiest practical criterion, it seems to us, 
a problem of major dimensions that is nonetheless carefully skirted in most discussions of 
educational indicators or management information systems in LDCs. The issue can be politically 
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sensitive, but it is also absolutely central to the development of any tracking system. The 
articles of Chapman (1990) and Chapman & Boyd (1988) are among the very few writings we 
have found to treat the topic straight-on, though the authors limit themselves to a particular 
technical treatment of the issue. 

The underlying question concerns the margin of error in data; and the basic principle to assert, 
we feel, is that all information contains elements of error, and any responsible presentation of 
data should include an estimate of this mai^ or at least a discussion of the sources of error. 
The underlying types of error are threefold, and can moreover compound each other: 
Reliability, validity, and sampling (representivity). Validity errors have already been discussed. 
We briefly consider the other two types in the following paragraphs. 

Reliability: Missing information, fanciful estimates, miscopying, falsifications, 
misunderstanding of instruments, and a legion of other miscues increase die margin of error in 
the data (or, technically speaking, increase the variation that would be found in the results of 
successive efforts to determine the same information by separate means). It is important to get 
some handle on this issue. 

In a certain sense, the problem may be compounded by the use of composite indicators.' (e.g., 
if our estimate of enrollments has a margin of error of +/- 10% and our information on 
numbers of classrooms has a +/- 20% confidence interval, the range for the pupil/classroom 
ratio computed from this data could vary by as much as +/- 29% around the true value.) 

Representivity or sampling error: What population do the data represent with reasonable 
accuracy? Oi^n data presented as national in scope have only been collected on a distinctly 
non-random subset of locations or units, perhaps because of poor response rates. This situation 
does not necessarily invalidate the information in and of itself. Decent data on a particular 
subset (which is usually representative of something, even if not of the whole country) is 
immensely better than no data, and may be very informative, provided that one takes account 
of its limitations. 

At first glance, it might seem impossible to get information on - and take account of - these 
sources and margins of error. Given that the whole topic has potential for undermining the 
credibility of any information system, one could then understand the inclination to scrupulously 
ignore it. But the main point is simply to refrain from throwing the baby out with the bath 
water; or, as the French say, to remember that "the best is an enemy of the good.- The main 
point is to start taking some account of sources and margins of error, to be ready to exclude the 
data that seems most suspect in this regard, and to begin pushing for inclusion of margin of error 
information along with any data collection or analysis activity* 

E. Whose Conception? 

A number of critical questions arise throughout the foregoing reflection on the conceptual 
framework and practical guidelines for devising an indicator system: Namely, whose conceptual 
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framework shall underlie the indicator system? how shall it be devised? and who shall specify 
the criteria for choosing indicators and the complementary roles of qualitative and quantitative 
inputs? Though there is incontestably an international language of educational planning and 
research currently largely dominated by economics, there are a number of reasons to assert that 
actual indicator systems may and should vary significantly by locality. 

The first reason is that, as noted above, efficiency has little meaning until operationalized by the 
specification of objectives, and those largely dq)end on the development strategy and 
perspectives of the countries in question. The second is that actual data availability may vary 
enormously as a function of other conditions within the host country environment. A third is 
that the ultimate utility of an indicator system depends entirely on one's ability to uiterpret the 
data, and most indicator systems therefore require a complementary set of qualitative insights 
that are normally highly country-specific. 

To the degree that there is effective host country participation, therefore, the potential for 
variation in the indicators from one country to another will significantly increase, even though 
there certainly are international paradigms of educational development that exert major 
conceptual influence nearly everywhere* Moreover, there is also a potratial here for variation 
over time. The notion of a key or critical aspect of an educational system has a time-bound 
quality and is strategy-related - aspects that are thought to be critical at one point in time may 
not be at another period. The high value (and rare occurrence) of longitudinal analysis should 
drive proponents of the system to resist too many changes, but the likelihood of variation in data 
specification over time must be taken into account. 

The conceptual analysis in and of itself therefore drives us to emphasize host-country 
participation and local ''ownership" of the indicator system design process. Of course, this shift 
in center of gravity does not automatically answer and resolve the •'cui bono?'' or •'who 
decides?" question. The same issues are simply transposed to a host-country level, and one must 
ask again whose criteria and conception are being used and to what degree they match those of 
the supposed beneficiaries or ultimate underwriters of the educational system. We will return 
to this topic a little further on in discussing field implementation strategy. 

m. DEVELOPMENT OF A PROTOTYPE MODEL 

A. Steps in the Process 

Our original intention was to move from consideration of these conceptual bases to the 
development of a simple indicator model applicable to all lEES countries and thence to collection 
and analysis of data that would illuminate trends in system performance and set a framework for 
discussion of Project impacts. As just explained, however, wc came out of the original 
conceptual stage with a different set of prqx)sed orientations. At that point, the ideal procedure 
would seem to have been an interactive design process including substantial host-country 
participation from the outset. For logistic and budgetary reasons, however, this was not 
feasible. Our fiallback strategy was to proceed with design of an initial prototype model, using 
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our collective field experience — and that of the rest of lERS staff - as a sounding board, then 
to treat the result as a stimulus for country-specific work and to develop a strategy for 
recentering the dd)ate in host-country locations. In this section of the paper, we briefly describe 
the model development process and its initial results. In the next section, we turn to the 
question of the process for recentering the debate in participating countries. 

The model elaboration process included the following steps: First, review of the theoretical 
bases of indicator systems and development of an initial conceptual framework. Second, 
inventory of potential indicators of system performance and classification into the categories of 
the conceptual framework. Third, choice of a set of indicators to compose the initial "indicator 
system." Fourth, submission of this framework to lEES staff and other colleagues for critique, 
discussion and modification. 

This work was undertaken in the course of the 1990-1991 academic year. Candidate indicators 
were drawn from Windham's monographs (1988a, 1988b), sector assessment documents. World 
Bank and UNESCO publications, the writings of Fuller (e.g., 1986, 1987) and others on 
educational quality concerns in developing countries, and experience with indicator systems in 
Indonesia, Haiti and Botswana. To simplify the conceptual task, we based all reflection on the 
example of a country's primary education system. 

B. Initial Conceptual Model 

The underlying concq)tual model that we adopted was based to a certain extent on work 
previously done in Haiti and consists of three tiers: Descriptive indicators, efficiency indicators, 
and equity indicators. At the descriptive level - portrayed on the adjoining table - the 
characteristics of the educational subsystem in question are organized into fifteen domains in 
roughly production-function fashion: Beginning, that is, with context and proceeding through 
inputs, processes, ouQ)uts and outcomes of education. In an effort at simplification, we decided 
to choose for the initial prototype model two summary indicators per descriptive domain - or 
thirty in all - based on criteria of likely availability and relevance to overall issues of 
educational quality and efficiency. It subsequently proved difficult to narrow choices down this 
fax without reference to the particular countries where the work would be done, so we retained 
an average of four proposed indicators per domain, counting on subsequent in-country work for 
refinement and pruning. The actual indicators and the rationale are laid out in Appendix A. 

The next tier of the model concerns efficieiicy indicators. Strictly speaking, efficiency is a ratio 
of outputs to inputs. An efficiency measure is therefore generally a secondary indicator which 
is based on two or more primary indicators and may span across domains of a model like the 
one presented above for the primary education subsystem. 

Each of the domains in the model may of course have its own internal efficiency measures, 
involving its own particular inputs and outputs. For example, the input to the school 
management domain may be dollars, and the output a certain number of services provided to 
teachers and students by school administration. These two measures could then be combined 
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into a ""local*" efficiency indicator concerning purely the realm of school management* For the 
most part, however, we are dealing with more global composite measures. 

Some of the descriptive indicators included in the list above are already implicit efficiency 
measures. Any form of completion or graduation rate, for example, is of this nature, since the 
""rate*" compares those who succeeded (output) to all those who took part (input). We propose 
in addition to track three kinds of composite efficiency indicators based on the descriptive 
indicators already detailed: 

L Enrollment ratios 

Essentially total enrollment over the size of the appropriate age cohort in the population. 

2. Resource sufnciency ratios 

These include textbooks per student and non-salary expenditures per student. 

3. Resource cost per graduate 

Actual costs per graduate are too complicated too compute and require cost studies to which we 
will probably not have access. A simpler measure is graduates per teacher-year. 

The third tier concerns equity indicators. Establishing equity indicators essentially involves 
attempting to disaggregate the types of data abready proposed for assessment of the overall 
primary school system. To keep from exponentially increasing the amount of work to be done, 
the number of equity indicators, and the number of axes of disaggregation, should be kept 
modest. We suggest three axes of disaggregation and four indicators for equity comparisons: 

Axes of disaggregation 

•Significant geographic or administrative regions 
•Private/public education 
•Gender (male/female) 

Equity indicators 

•Net enrollment ratio 
•Cycle completion rate 
•Textbooks/student 
•Teacher training 
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C. Debate and Modification 

Discussion of this overall model with lEES Project staff proved to be a highly instructive 
exercise and yielded a number of suggestions for improvement. Notable among these was the 
recommendation that at both the central and the country levels a distinction be made between 
an '•ideal*' version and a "feasible" version of the model 

The underlying idea is to identify and distinguish both a "best case" and a "likely" set of 
indicators. The first one gives all participants an occasion and an incentive to think about the 
kinds of information that they would really lite to have in order to inform educational decisions 
bearing on system efficiency. It also gives us some protection against the danger of getting 
mired down in "least common denominator" approaches that lead to collection of an incoherent 
assemblage of data with borderline validity simply because it was (all that was) available. 

The second approach (development of "feasible" indicator sets) provides a reality check against 
impractical or inappropriate model q)ecifications bom of too much concern with conceptual 
elegance or completeness and insufficient attention to what is currently possible in different 
country settings. We anticipate that iterative review of the ideal and feasible models will result 
in (a) an operational (feasible) model that is conceptually tighter, and (b) a better articulated 
optimal model that will be increasingly useful for purposes of training, policy discussion and 
educational planning. 

IV. IMPLEMENTATION IN THE FIELD: PROCESSES ENVISAGED 
A. Development of country-specific models 

The next stq) foreseen in the process is to move the center of gravity in the development of the 
ideal and feasible indicators sets to three TEES participating countries. At present, there have 
been expressions of interest, but the final "self-selection" of these countries has not been made, 
nor has the exact process to be respected in-country been specified. This will doubtless vary 
from one location fiom another and be developed largely in situ, with the participation and 
support of lEES Resident Technical Assistants, where applicable. The minimal objective of the 
exercise will be to use the proposed system simply as a tool for discussion and staff training, 
and to enlist the help of host country counterparts in modifying and improving a model that the 
lEES Project will then use to track trends in educational system performance in the country. 

If there is interest in participating countries, a good deal more can be done with the resulting 
system, or with the process of developing it. The exercise can s^e as a basis for reviewing 
current uses of educational management information systems, designing means for interpreting 
their outputs, integrating qualitative information into the interpretation process, specifying types 
of local process research to be targeted for support, and/or training staff at different levels in 
the related analytic tasks. 
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B. Related Issues 



Four major issues that merit exploration in tandem with this process are worth mentioning here. 
The first concerns the complementary roles of qualitative and quantitative outputs in educational 
decision-making, and the best means for ensuring a supply of relevant qualitative insight. 
Process data is the habitual weak point of indicator systems. We therefore envisage, as one 
component activity, a careful census of the qualitative and process studies currently underway 
in the countries in question (and potentially the ^couragement of others through the lEES 
research agenda) in order to fill the gap and lay a firmer foundation for analysis. 

A second and closely related issue concerns investigating actual patterns of use of data in 
educational agencies and pinpointing factors that determine their utility. The "sociology" and 
politics of data use is a topic as centrally important to EMIS operations as it is sensitive, and 
certainly merits thoughtful recognition. 

A third important topic is related to the ^propriate mix of nationwide and sample data in an 
indicator system. A number of more refined quantitative indicators - as well as most types of 
qualitative information — cannot be reliably collected for the whole country but can be very 
usefully approximated through focused sample studies. Developing this methodology as part of 
an EMIS merits closer attention. 

A fourth critical subject concerns the relationship to be developed between national management 
information and indicator systems on the one hand, and regional or local ones on the other. Our 
experience clearly suggests that overly-centralized systems where field staff collect data solely 
to feed central file*: or computers end up receiving increasingly unreliable information. It is 
important to develop at the same time methods and procedures for synthesis and use of data at 
local and regional levels, not only as a support for better decision-making at this level, but also 
to help ensure that field staff see the use of the data collection effort and remain conscientious 
about it. 

C. Analysis Strategy 

Analysis is another congenital weak point of indicator systems, as participants in the current 
debate about educational indicators in this country have been quick to point out. Exactly how 
is one supposed to use the data gathered? Here lies one clear advantage of trying to articulate 
an underlying conceptual model, as we have begun to do above. That model can serve as an 
important aid in the analysis and interpretation of the indicator data. 

Generally, we see two related directions for analysis: (1) examination of trends over time; and 
(2) diagnostic interpretation of system status data. 
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L Examination of trends over time* 

Part of the purpose of the indicator system is to provide a means of tracking the evolution of 
the educational system (most fundamentally vis-a-vis lEES activities) and spotting changes that 
provide early warning of serious problems or early evidence of major opportunities for school 
improvemrat. Consequently, a first level of analysis consists essentially of dedudng direction 
and magnitude of changes in the key indicators fh)m the longitudinal data assembled. 

This is, of course, not quite as simple as it sounds, even assuming that enough sequential data 
on the same indicators are assembled to document a trend. One problem that typically 
confounds such an effort must be resolved before analysis can begin: The non-comparability 
of data on the same indicator at different points in time. It is quite conceivable that over time 
the indicator may be defined differentiy, the population on which the data were collected may 
be different and/or the reliability of the methods used may vary markedly. An initial effort to 
take account of these irregularities will need to be made. 

2. Interpretation and diagnosis 

In a sense, the analytical model pictured above already constitutes a framework for interpretation 
of the data and even diagnosis of the problems observed, since the network pictures a series of 
supposed causal linkages among educational system variables. If an anomaly or problem is 
observed at one level, we therefore have at least a suggestion of where to look for causes or 
contributing factors. 

Of course, what constitutes an **anomaly** or ""problem*" for highlighting and investigation can 
only be determined with reference to the particular objectives and norms of the educational 
system under examination. This once again supposes the active participation and involvement 
of host country planners. 

These remarks suggest the following order of action for interpretation of the data: First, 
determine trends. Second, isolate trends or persisting conditions that seem problematic, given 
the country's educational objectives (or, in their absence, accepted international norms.) Next, 
by comparative trend analysis, formulate questions and proposals about possible contributing 
conditions. Fourth, discuss the entire matter with host country researchers/plannen and revise 
the analysis on this basis. Fiftii, revise the indicator scheme and collection plan as warranted 
by the results of the analysis. 

D. Dissemination Phase 

The efforts described in this paper seem to us potentially to have some larger implications for 
data collection practices in education at the level of each country, as well as internationally. As 
much should be expected, since the lEES Project is in part an R&D endeavor with the explicit 
charge from AID of developing new procedures for educational planning and research that may 
be worthy of broader dissemination. If so, how should such dissemination be handled? 
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We think that the first critical dimension of the dissemination task lies within each host country. 
We will therefore be verifymg in each case what are the existing in-country media for 
dissemination of educational research and reflection, and how we might collaborate with 
counterparts in using - and simultaneously in strengthening - these media. A second dimension 
evidently concerns exchange among participating countries and/or with educational personnel 
from potentially interested nations in the same region as the host country. A third and final 
dimrasion embraces international dissemination and publication within the United States. 

In addition, we oivisage publication at one or more of the above levels of a series of 
monographs to ensure broader availability of documentation on the development and products 
of this activity. Each monograph will represent a phase or particular theme of the project. Four 
distinct monographs and topics are presently anticipated: 

(1) Overview monograph on concqption, development and adoption of appropriate 
indicator systems for tracking educational efficiency; 

(2) Survey and analysis of research now completed or presently being conducted in 
participating countries on educational processes and quality dimensions, the habitual 
weak areas in indicator systems; 

(3) Monograph on problems of dovetailing national and local indicator systems and MIS's 
and on the staff and teacher training issues involved; and 

(4) Monograph on questions of cross-national comparison of procedures and resulting 
data from efficiency indicators systems. 

V. CONCLUSIONS 

Several limitations of the analysis and processes presented in this monograph should be kept in 
mind. The first and foremost is that the schema presented is purely and simply a framework for 
thmking about indicator systems and so a potential starting point for the more important phases 
of related work that must go in the field. A second limitation is that the discussion is largely 
theoretical. In fact, much of what needs to be done in improving data utilization in the field is 
more particular and concrete and has to do with questions of how useful insights are drawn from 
existing data, how one diagnoses the small shifts in the type of information collected that could 
have major impacts on its usefulness, how disparate data collection systems m a single country 
or region are coordinated to eliminate waste eiSbrt and contradictions, and so forth. The 
framework suggested in this paper may nonetheless have some value for stimulating discussion 
and focusmg attention on some ''common denominator" issues in mdicator system management. 

In summary, one could say that ''a funny thmg happened on the way to establishmg a 
project-wide indicator system for tracking efficiency trends": We remembered the importance 
of place and process and came up instead with a plan for engaging host-country participants in 
dialogue about - and collaborative design of - more flexible and location-specific systems. 
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That this should happen in the TF.RS Project is not surprising, for some of the Project's finest 
hours and best accomplishments have lain in the impulse to translate the current technique and 
jargon of educational planning into participatory exercises, accessible to host country 
counterparts and subject to their modification, criticism and influence* Sector assessment as 
practiced in the project, for instance, for all its sometimes heavy jargon, positivistic assumptions 
and economic bias, has been widely appreciated as a means of making the process of foreign aid 
allocation and educational policy analysis more visible and of opening the door to greater 
direction of the process by host country nationals (Easton, 1988). A number of project activities 
have had, intentionally or not, major institutional devdopment consequences; and most have 
been characterized by a concern to adapt research agenda to capacity-building priorities. The 
reorientation of the efficiency trends activity is thus in a sense simply a move to make it more 
consistent with the spirit of the TEES Project- 
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APPENDIX 



DEFINITION OF MODEL DOMAINS WITH PROPOSED INDICATORS 
I. CONTEXT 

1. Socio-Ecommic and Demographic Background 

This domain serves tc establish the baseline context for assessing schooling inputs, 
processes, outputs, and outcomes. Five indicators have been selected to represent this 
domain: a) total population, b) life expectancy at birth,c) Purchasing Power Parity GDP 
per capitei, d) newspaper circulation per 1000 population (to reflect 
literacy/communications), and e) the number of adults completing primary education. 

n. INPUTS 

2. School Endowment 

The domain of school endowment is intended to reflect the basic physical and material 
resources available at the school site. The three indicators selected to reflect the 
availability of instructional resources are a) class to classroom ratio, b) whether furniture 
is provided, and c) the availability of instructional materials. 

3. Local/Community Support 

The degree to which local schooling is supported by the community is reflected by this 
domain. The indicators of this support are a) whether a functional PTA exists, b) the 
nature of school-community contacts, and c) the percentage of school expenditures met 
through non-central government sources. 

4. Central/Regional Support 

The support offered to local schooling through the central/regional offices of the Ministry 
of Education, or other institutional provider is represented by a) the number of textbooks 
produced, b) the number of school inspection visits by MOE officers, and c) the overall 
and non-personnel Govemment/MOE expenditures. 

5. Student Characteristics 

Student population characteristics, i.e., what they bring to the schooling enterprise, are 
represented by a) total enrollment, b) the percentage of female students, and c) data on 
their health and nutritional status. This domain of student characteristics is distinct fi-om 
how students engage in the instructional process, which is represented below in "student 
participation." 
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6. Teacher Characteristics 



The characteristics of the teacher population (status of the profession and what they bring 
to the instructional process) are represented by a) the number of teachers, b) the 
percentage of female teachers, c) training (by level), d) average salaries, and e) attrition 
rates. 

m. PROCESS 

1. School Managemem 

School management refers primarily to the organization and provision of instructional 
time and resources, with additional indicators of administrator qualification and 
efficiency. To represent instructional time and resources, the following indicators have 
been selected: a) number of school days per year, b) hours of instruction per day, and 
c) school expenditures on facilities and maintenance. Indicators of administrator 
characteristics are d) the percentage of headmasters completing headmaster training, and 
e) a summary index of administrative efficiency.* 

8. Curriculum Quality 

The quality of the curriculum will be represented by summary indices of a) the degree 
to which the national curriculum is followed, and b) the appropriateness of the national 
curriculum.^ 

9. Teacher Quality 

The quality of teaching is represented by a) the amount of teacher's time spent on 
instruction, b) the use of instructional materials for instruction, and c) the frequency of 
testing and feedback to students. 

10. Student Participation 

This domain refers to effective school attendance and participation in organized learning 
activities. The underlying variables concern quantitative and qualitative appreciation of 
••time on task** - that is, time spent on the activities designed or intended to promote 
achievement of the school's learning outcomes. The indicators that we have chosen in 
this realm are a) student absenteeism rate and b) student time on-task. 



^Noto: ThMt 'indtcaton* art a conceptual pUcemark, repceaentiag, to date, caugoriea rather than actual indicaton. 
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IV. OUTPUTS 



IL Student Attainment ofid Achievement 

This domain concerns the degree of achievement of primary school learning objectives 
in the cognitive and psychomotor realms and includes quantitative and qualitative 
assessments of those scholastic "outputs/ The indicators that we have chosen in this area 
are a) percentage of students reaching the last grade of the cycle; b) results of mastery 
exams; and c) student grade-to-grade progression rates. 

12, Student Attitudes and Aspirations 

This domain concerns the degree of acquisition of selected attitudes, or the degree of 
achievement of specified affective learning outcomes, by primary school students. Chosen 
indicators are a) percent of students taking admissions exam for next cycle and b) the 
general area of student occupational choices (specific indicator yet to be designated), 

V, OUTCOMES 

13, Later Academic Outcomes 

••Later academic outcomes'* refers to the degree of success of primary school graduates 
in their further schooling, including admissions to higher level institutions of learning and 
completion of these subsequent cycles. The specific indicators chosen in this area are a) 
percent of graduates of the primary cycle entering the following cycle and b) the number 
of graduates in scientific/technical fields, 

14, Economic Outcomes 

The category "economic outcomes" refers to financial success in post-schooling 
employments and can include indicators like measures of the length of job search, 
stability of employment, and income of primary school completers or leavers. The 
specific indicators chosen are a) average earnings of graduates of cycle and b) labor force 
status of graduates of cycle. 
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15. Social and Political Outcomes 



This domain covers other dimensions of social and political behavior that may be 
influenced by primary school attendance, including social mobility, political participation, 
number of children and their schooling, etc. The indicators chosen for this exercise are 
a) the general area of political participation of graduates of cycle (particular indicator yet 
to be designated) and b) an mdex (yet to be specified) of social mobility of graduates of 
cycle. 

16. (Feedback to Context) 

This category simply mdicates that the outcomes of primary schooling have in turn an 
impact on the characteristics of the socio-economic context in which the school system 
operates. 



22 

30 



2 



The Role of Educational Management Information Systems 
and Indicators in the Operationallzation of the Concept of 
Educational Efficiency: Eight Years of lEES Project Experience 



Dwight R. Holmes 



It is now nearly eight years since the DEES Project began in 1984— lEES being a ten-year 
initiative dedicated to assisting several developing nations to Improve the Efficiency of tiieir 
Educational Systems. Note that you can see that what my colleague Peter Easton enjoys saying 
is true: "Efficiency is our middle name." Given that we are eight-tenths of the way there, it 
would seem incumbent on us at the project— and instructive for all of us in the field— to reflect 
back on the TEES experience, and to see where this pursuit of educational efficiency has brought 
us— to trace the evolution of the concept of "educational efficiency" and its companion concepts 
"Educational Management Information Systems" (EMIS) and "indicators," as they have 
developed and evolved within the project itself. This paper is a brief synopsis of my ori-going 
research on this project experience, based primarily on lEES documents and reports. 

For those not familiar with the project, I should point out that it has participated in EMIS, 
planning, and research activities in several countries, including Yemen, Somalia, Botswana, 
Nepal, Haiti and Indonesia, among others, and that the majority of the activities have been very 
country-specific, and not necessarily related to the work going on in other countries. More 
about the ramifications of this later on, but this is just to point out that it would generally seem 
more appropriate to speak of "TERS experiences" in the plural rather than the singular. 

Efforts to improve the quality of education (following unprecedented growth in school 
enrollments worldwide) were coincident with new and stringent requirements to contain costs 
in the wake of the oil shocks of the 70's and recession/debt crisis of the early 80's. It was in 
this context that "efficiency" assumed its prominence as a goal and theme among donors and 
agencies involved in development education (Chapman & Windham, 1986). The mandate of this 
project was to assist countries to mal^ more efficient use of the resources already available for 
education, instead of planning new, add-on programs which would require increased spending. 
Related to this approach were five issues, identified in the 1983 project proposal: 1) external 
efficiency (relevance); 2) internal efficiency; 3) access and equity; 4) administration and 
supervision; and 5) costs and financing (lEES, 1983). 

Efficiency in education is discussed in the lEES I Proposal (and later in Windham's monograph 
and consistently throughout the project) in terms of external and internal efficiency. External 
efficiency refers to the relationship between school and work: How relevant is the product of 
the education system (its students and what they learn) to the world of work— and 
leisure— beyond school? The schools are approaching external efficiency to the extent that they 
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are preparing their students for life beyond their school-going years* Internal efficiency, on the 
other hand, is defined as a ratio of the inputs put into schooling— the available resources— to the 
ou^ut of schooling— the desired educational outcomes* Chapman & Windham, in their 1986 
lEES monogr^qph on ""Evaluation of Efficiency in Educational Development Activities'* point out 
that you can have both school quality and school effectiveness without necessarily being e£Rcient 
in doing so. The process is, then, to identify the desired ou^uts, the goals of your education 
system (assume for the moment that you not only are able to articulate those goals, but to 
specify and measure them validly as well...); to the extent those goals are maximized, you have 
realized school effectiveness. You thra compare those ouQ)uts to the inputs that went in, your 
costs. The resulting ratio is your measure of internal efficiency. And thus, as Windham writes, 
"efficiency subsumes effectiveness (Windham, 1988). " 

The criticism is often heard that, in education, the pursuit of efficiency is done only at the peril 
of the pursuit of quality. Ch^man & Windham state, however, undoubtedly in anticipation of 
this argument, thdt "efficiency is a criteria of instructional goal attainment, not an instructional 
goal in and of itseir (1986, p.4). The question is therefore not "Have we attained efficiency?" 
but "Have we achieved our educational goals, and done so as efficiently as we might have?" 
They pointed out that the instructional goals must be specified by aU of the stakeholders 
involved— family, students, voters, policy- and decision-makers... And they criticize those who 
design oversimplified production functions which concentrate on some measure of academic 
achievement for ignoring the fact that education is a multi-input, multi-output activity. So that, 
^ for example, using test scores as the desired outcome and measure of school e^ectiveness 
ignores other competing and often contradictory goals such as distributional equity, and 
attitudinal and behavioral changes, to name two. 

The logical extension of this would be, then, that ultimately each district, school, or teacher has 
to make their own decision about what goals to optimize and give priority to. 



In what is perhaps the most debated position in the Wmdham monograph on "Indicators of 
Educational Effectiveness and Efficiency", it is stated that "the efficiency concept is a neutral 
device" (Windham, 1988, p.7). Taken out of context tiiis may sound like a call to rally behind 
the cause of positivism and the objectivity of science. But Windham continues on to say that 
the success of the project in defining and measuring the inputs, processes, ou^uts and outcomes 
of the educational system will "determine whether the current attention focussed on efficiency 
is a positive or negative contribution to educational development" (Windham, 1988, p.7). 

Thus we can say tiiat if tiie efficiency criteria is controversial it is not for tiie reason that some 
would desire to waste educational resources; instead, it is because there are legitimate 
differences about what constitutes the nature of the teaching-learning process, and what tiie 
desires outcomes of tiie system should be (Chapman & Windham, 1986). The goals are 
contestable, and the measurement of the components is problematic. Smce the lEES Project has 
been involved in data-based decision-making it has had to contend with both the contesting of 
goals and the problems of measurement. More about tiiese aspects below. 
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The project's central concern is efficiency in education, and we have briefly reviewed how that 
has been defined at the theoretical level. Let us now move on to consider at how the project has 
tried to bring about these improvements. 

At the heart of the TRRS Project philosophy is the use of the Education & Human Resources 
Sector Assessment, a strategy which grew out of Florida State University's experience working 
with the Korean government to reform its education system in 1971 (Morgan, 1990). A sector 
assessment team consists of TEES experts and local educationists who work together for a period 
of six to eight weeks, to collect and analyze all available, relevant information and data about 
each aspect of the country's educational system— primary, secondary, tertiary, 
vocational/technical, adult and non-formal, and so on, in the context of the economic situation 
and taking into account the country's management capacity. The team attempts, through this 
process, to identify priority areas for investment in education, and to focus attention on the 
constraints which are present (DEES, 1985). Their recommendations are targeted to improve the 
utilization of financial and human resources within the education sector. The sector assessment 
philosophy is, then, to replace the "project s^roach*" to planning (which had predominated 
earlier) with a comprehensive, "systems" orientation, in order to mate more efficient use of 
available resources (lEES, 1985). The resulting document—tiie Sector Assessment—establishes 
the structural and institutional baseline of a country's education system, and points to a variety 
of EMIS, planning, research and project activities which the concerned Ministry may elect to 
incorporate into its workplan for the years ahead. 

The goals of this Sector Assessment procedure are 1) to enable decision-maters in the Ministry 
to assess the status of educational development within the framework of national goals on an on* 
going basis; and 2) to aid them in targeting s^ropriate interventions which would bring about 
efficiency improvements in the system. This process also helps to accomplish the third goal of 
the project, which is to strengthen local capacity in educational planning and management (lEES, 
1985). 

The innovative aspect of this sector assessment approach was that it was to be "process 
oriented"— it would focus on putting in place continuous operations to measure "objectively 
verifiable education and training outcomes" (Florida State University, 1983, p. 36). It would 
not focus on any one structure (such as school buildings, teacher training, curriculum reform or 
instructional materials). Nor would it be guided by the idea that any one solution had universal 
validity (such as instructional technology or the use of media, literacy campaigns, textbook 
distribution, etc.). All of this deriv^ from the observation that education is a multi* 
input— multi-output phenomenon (Florida State University, 1983). 

As indicated above, it was intended that the sector assessment process would be an on-going 
one. Each Education & Human Resources Sector Assessment was designed to be updated as 
necessary, with increasing participation and direction by host country personnel over time 
(Rorida State University, 1989). The long range objective was for the host country to gain the 
capability of conductmg its own on-going assessment (lEES, 1985). Whereas the "Sector 
Assessment serves as the initial version of a continually updated resource to guide all activities 
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in the educati(Hi sector"* (TRRS, 1985, p. 16), the updates would be annual data-gathering 
activities which arc topical in scope, not comprdirasive. This structure would be the ''long term 
vehicle for establishing and demonstrating the value of data-based planning, while creating 
opportunities for the development of host country cs^ilities*' (IBES, 1985, p. 16). And 
together these assessments and updates would form the core of the project data collection and 
analysis activities, which would in tum serve as the basis for efficimcy improvements and for 
establishing policy dialogue based on data-based decision-making (I^S, 1985, p. 15). This 
proposal to instituticmalize the sector assessments approach as a regular activity has not been 
successfully carried out in any of the lEES countries. (Sector assessment updates were carried 
out in Botswana and Indonesia, but not in the annual, routinized manner foreseen in the project 
proposal.) 

In addition to the proposal for annually carrying out sector assessment updates, an entirely 
separate strategy was being devdoped within the project as well, called Educational Policy 
Analysis Training. This activity was initially jointly sponsored by lEES and the World Bank, 
and was developed originally for use in Togo in 1986, in cooperation with a West African 
university consortium (Milton, 1986). The approach was to involve participants in the 
formulation of policy questions and alternative proposals with the use of case studies, which 
were designed using actual data collected in Botswana. Participants were given hands-on 
instruction in some statistical procedures, construction of data tables, and critical reading of the 
same. As part of the articulation of policy questions and responses, they were also given 
instruction in cost/beneiit types of analysis. 

This Educational Policy Analysis Training was in ^t quite successful in its initial run in Togo. 
Materials were produced by the lEES team for both the participants and facilitators, which 
contained the case study materials and sequenced instruction, organized according to this flow 
model (see Figure 1). 
The units were: 

1. Understand the problem; 

2. Plan the analysis; 

3. Describe the current situation; 

4. Restate the problem; 

5. Generate alternatives & narrow them down; 

6. Analyze costs; 

7. Analyze the consequences of alternatives; 

8. Rank alternatives; 

9. Prepare the final report. 

In analyzing the reasons for the success of this approach, and what its underlying philosophy 
was, it would seem it provided participants from the ministries with practical work in 
statistics— in contrast to what is offered in typical university courses in statistics— while also 
providing them with the opportunity to develop and practice skills in dealing with issues of data 
quality and other caveats. In addition, it gave them practice in interpreting data while utilizing 
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their own knowledge of the educational, social and political context in their countries. The use 
of real data proved to be very successful, as the West African participants were highly motivated 
in their discussion and analysis of policy alternatives (Milton, 1986). 

The implicit philosophy of the training was this: For both the data technocrats, who believe that 
data is infonnaticm, and the data disdainers who believe no data, this was a piacticum in the fact 
that data is not information, but can, when used judiciously, provide a meaningful basis for 
decision-making. 

An important question is, then, what h^jpened to this training module, after it was tried and 
tested in Togo? It speared to offer a methodology for developing local capacity for data-based 
analysis that was both relevant and could work. This should have, in turn, resulted in increased 
efficiency in the education sector. Why was this approach not pursued further? Several factors 
seem to have contributed to the feet that this package has remained essentially "on the 
sheir— factors both exogenous and indogenous to the project. 

In feet, it was planned to use the Educational Policy Analysis Training materials for a training 
in Haiti, and arrangements had been made for this in the summer of 1987. Unfortunately, the 
day of the first session the streets of Port-au-Prince were fiUed with gunfire, no one from the 
ministry showed up, and the BEES team which was there to conduct the workshop had to leave 
by the next plane out. The workshop was never rescheduled, as that was the last lEES activity 
with the ministry in Haiti. (Most lEES work in Haiti has been with non-government schools,) 
Additionally, the World Bank decided against continuing in its role with the activity, which is 
the reason why the planned second running of the workshop in Togo never took place. 

More essential, however, is the fact that although DSES is a centrally-funded project (meaning 
that it funding and direction come out of US AID/Washington and not from the US AID missions 
in cooperating countries), the activities are largely independent country to country, and, for tiie 
most part, decisions are made between the local AID mission, the concerned Ministry, and tiie 
lEES country coordinator in Tallahassee. Only those activities falling within a particular 
country's Country Plan, are implemented. There is, for the most part, no central direction in 
the devdopment of these plans. While some would argue that this is by design and concurs with 
lEES' overall decentralization posture, others might say that the project lacks needed conceptual 
direction. And furthermore, that it suffers from bureaucratic amnesia. The Togo materials are 
quite likely not the only ones sitting on the shelf in Tallahassee which might be of good use in 
one of the project countries. 

Of course, the fact that the Togo package of materials has not been specifically incorporated into 
workshops elsewhere does not necessarily imply that training in policy analysis Im not been 
conducted under TF.RS. In fact, much of this same kind of work— training in policy 
analysis—has been going on continuously on a day-to-day basis, with the project advisors and 
their various counterparts and colleagues. And this may ultimately be as effective a means of 
imparting that type of training as any. Being less formal, it tends to remain less documented 
as well. 
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The first few years of EMIS work had been ^nt introducing computer hardware and software, 
revi^g school report forms along the lines recommended in Sector Assessments, and developing 
the ministries* in house cs^ity to collect and rqport statistics in a timely manner. All of which 
were precursors to being able to do any real policy analysis, on the one hand, or to inform the 
planning process, on the other. As this EMIS work progressed along, there were advisors and 
consultants working in close collaboration with education ministry personnel in their efforts to 
improve the usefulness of the data being collected; and this created the need to operationalize— in 
very specific and practical terms-^such concqpts as educational quality, equity, and efficiency. 
Project personnel were forced to move from the more theoretical formulations of early 
documents on efficiency indicators to the nitty-gritty of constructing composite indicators from 
available data. This enabled the focus of the EMIS systems to progress from data production, 
the focus of the first five years of the project, to poUcy formulation, the focus of the second 
phase (Florida State University, 1989). 

Much of this work was carried out in Indonesia by Jim Cobbe, an economist. His tasks were 
to construct, in conjunction with his counterparts, a system of educational indicators to be used 
in policy luialysis and educational planning at the national and provincial levels, and to 
collaborate in identifying and devdoping a series of indicators of educational quality, equity and 
efficiency which would serve as a basis for a resource allocation model for financing the 
education sector. The £act that the Ministry of Education & Culture made efforts to requested 
assistance in carrying out this specific task signified that the ministry had realized it would 
benefit from having indicators of quality, equity and efficiency available to use in its lobbying 
efforts with the Ministry of Finance during the annual budget negotiations. 

Cobbe pointed out that efficiency measures, in the present Indonesian context, are necessarily 
indirect, since there exists neither a baseline of entry-level standards and achievement data nor 
any definition of the education system's concrete objectives (Cobbe, 1990). Given that, what 
you are left with, he says, is the assumption that quality is somehow constant, and therefore that 
if the ratio of inputs to outputs goes down, there has been an increase in efficiency. Somewhat 
contradictorily, you are also left with the assumption that if the ratio of inputs to ou^uts goes 
up, there must have been an increase in effectiveness. It is because of this paucity of data and 
agreed-on objectives that efBciency indicators appear to be the inyerse of some quality indicators 
(Cobbe, 1990). He then goes on at great length to caution us that interpretation of indicators 
must be done by analysts who know what they are doing and why, and who are very familiar 
with the sources and nature of the data which they are analyzing; indicators are, at best, 
imperfect and mdirect, and can be greatly misleading. 

The immediate output of this activity was a manual on the production of analytic educational 
indicators, showing which statistics are available where, and how to calculate useful indicators 
from them. It also lists various caveats about the weaknesses of the same. 

At a more analytical level, and growing out of deliberations among TEES core siaff members, 
the need for a framework of indicators was realized. Ensuing work led to the development of 
an indicators model, which was intended to provide the framework for an indicators system 
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within an EMIS. The initiai phase of this activity, carried out as a home office activity, is 
outlined in the report entitled "Collaborative Design of Educational Indicator Systems in 
Developing CcHmtries" by Easton et al., which is included in this volume. Ad^tation and 
implementation of the model is presently being carried out in two lEES countries, Nqnl and 
Botswana. The chs^ters by Williams and Burchfield report on those activities. The 
developmrat of the modd itself is briefly described in the following pages. 

First, a literature review <hi the theoretical zspccts of indicators and indicator systems was 
carried out, followed by the compilation of an inventory of indicators reported or discussed in 
a variety of U.S. and international sources, including lEES project materials such as sector 
assessments and the Windham monognaqph. Altogether over 600 indicators were listed, ^^^lich 
were as specific as "the per student per cycle cost of primary education** and as broad or vague 
as "teacher quality" and "articulation of the curriculum." Some w^ ambiguous, sudi as 
"student participation. " These indicators were then sorted into categories which were eventually 
labeled "domains" and placed into the model (Figure 2) which drew on earlier work by Odden, 
and the IIBE Project in Haiti (Odden, 1990; Easton et al., 1990). 

The final and most significant step is underway at present: The adaptation of this model by 
participating ministries in Nepal and Botswana* There are several reasons why indicator systems 
should be locally generated (Eastern et al., 1990) including the fact that data availability varies 
greatiy across countries; periiaps tiie most important reason is that highlighted by Windham 
earlier: Efficiency has littie meaning until operationalized by the specification of objectives, 
which must be country specific. But note that shifting the process to host countries does not 
make the question "Whose goals? and who decides?" any less relevant. 

The lEES efforts to operationalize the efficiency concept led to a focus on indicators; and the 
need to maintain the systems s^roach and to provide a frame of reference for choosing and 
analyzing indicators led to the devdopmmt of this model. 

The underlying concq)tual model consists of tiiree tiers: descriptive mdicators, efficiency 
indicators, and equity indicators. At the descriptive level (see Figure 2) the characteristics of 
the educational subsystem in question are organized into fifteen domains (e.g., teacher 
characteristics, student participation, etc.) which are, in tum, grouped within tiie aspects of 
context^ ir^utSy processes^ outputs and outcomes of education. While some of die suggested 
indicators are purely descriptive (e.g., number of school inspection visits), others are in fact 
ratios and thus are themselves impUdt efficioicy measures (e.g. , percentage of students reaching 
last grade of cycle)— which is to say that tfie model is ndtiier fully developed nor consistent. 
At the second tier of efficiency indicators, some efficiency indicators were specifically proposed, 
such as "resource cost per gnuiuate." 

The third tier concerns equity indicators* Establishing equity indicators essentially involves the 
disaggregation of tfie data already proposed for use in assessing the overall school system. In 
an effort to keep the size and complexity of the system manageable, it is proposed that the 
number of equity indicators, and the numt)er of axes of disaggregation, should be kept modest. 
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Easton et al. have suggested three axes of disaggregation and four indicators for equity 
comparisons (also see Figure 3): 

Axes of disagsregatioii 

•Significant geographic or administrative regions 
•Private/public education 
•Gender (male/female) 

Equity indicators 

•Net enrollment ratio 
•Cycle completion rate 
•Textbooks per student 
•Teacher training 

Above all, it is emphasized that indicators do not explain anything; they only point (Bottani, 
1990). All importance must be given to the interpretation of the indicators, so that we know 
what is being pointed to. 

The intent behind the development of this model was to establish a frame of reference for 
selecting appropriate indicators, and a process which might enable the development of durable 
EMIS indicator systems in different countries. 

The model then serves two functions: It is a template which can assist planners in sifting 
through vast amounts of data to select and prioritize a manageable number— and representative 
range— of indicators to monitor; it also provides a structure of plausible comparability across 
countries (whereas the specific data may not themselves be comparable, the framework of 
analysis is). The model enables countries to select indicators of efficiency in aspects of the 
system consistent with the direction of national goals and priorities while still keeping the whole 
system within view. This model is being utilized in Nepal and Botswana— these experiences are 
reported in other chapters in this volume. The utility of these efforts to measure •'educational 
efficiency" for the purpose of providing feedback on the performance of the education system 
and informing policy debate will need to continue to be assessed beyond the life of the lEES 
Project. 

CONCLUSION 

Under the aegis of EMIS, research, and indicators activities, a variety of strategies have been 
utilized to build the infrastructure and human capacity in participating ministries for data-based 
decision-making. Computer hardware and software was brou^t on, and training provided to 
appropriate personnel; school statistics forms were revised, and collection processes streamlined. 
In some countries, research was carried out on the subject of how officials regarded and used 
the data that are available, and how reliable the data are. Modules were developed and tried for 
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training ministry officials in utilizing data for policy analysis— though that particular has not been 
used again. Nevertheless, other trainings have bera given to those in planning and management, 
and considerable on-the-job training was gained by those who have worked with TEES resident 
technical advisors. Recognizing the need to specify and quantify educational inputs, processes 
and ou^uts, the Windham indicators monogrs^h was produced, and considerable work was done 
in Indonesia, in particular, but also Haiti and Botswana, in defining sets of indicators that would 
be both practical to produce and meaningful to interpret. This was followed by development of 
the indicates systems model by Easton et al., which is designed to serve as an aid in both 
setting up ENflS/indicator systems, and in formulating policy questions focussing on issues of 
efficiency and equity. 

Though perhaps none of the countries is now in the situation of having ready-made "efficiency 
report** schemes in place, one can see considerable movement from the crude to the subtle if one 
compares the early work in building up the statistics collection curabilities with the current work 
underway in sev^:al countries* 

One could argue, of course, that the experience of the project has been one more focused on 
rationalizing educational planning and policy-making than on directly impacting the efficiency 
of the educational systems. The actual sphere of influence of the project may in fact be at the 
level of working towards more ^efficient" decision-making within the structure of the Ministry, 
and is still several levels removed from directly influencing the more efticient allocation of 
resources, let alone the efficiency of operations in the schooL Perhaps Windham's warning was 
well-placed: The unresolvable problems of articulating a set of goals for the system which all 
or even most can agree on, coupled with the technical problems of how to explicate, 
operationalize and measure inputs, processes and outputs of the system may render useless the 
efficiency concept so &r as educational policy analysis goes. 

Definitely, data and data-based analysis have begun to more frequently inform the decision- 
making processes in some lEES countries. But are the Ministries of Education making decisions 
now that lead to greater efficiency in the school system than was previously the case? Certainly 
there are some specific examples which can and have been cited. But systematically, is this the 
case? The evidence for this has not been documented. Given the continuing flux in economic, 
demographic and political situations it would be surprising if it had been. 
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3 

PLANNING AND IMPLEMENTING EDUCATIONAL MANAGEMENT 
INFORMATION SYSTEMS: THE CASE OF BOTSWANA 



Shirley A. Burchfield 



BACKGROUND 

OVERVIEW OF THE EDUCATIONAL SYSTEM 

Situated in Southern Africa, Botswana is about the size of Texas (582,000 square kilometers), 
with a population of 1.3 million in 1991 and a Gross Domestic Product of U.S. $2.4 billion. 
Per capita income is U.S. $1,050. The primary school net enrollment ratio is 92%, and the 
proportion of Standard 7 primary students who go on to junior secondary school is 81 %. The 
structure of the educational system is currently 7 years of primary, 2 years of junior secondary 
and 3 years of senior secondary education. The Ministry of Education is responsible for the 
administration of the educational system. It sets and marks Primary Leaving School Leaving 
Exam and the Junior Certificate Exam, as well as recruits and deploys the teaching staff in 
primary and secondary education. 

In order to carry out its managemmt responsibilities, the Ministry maintains Departments of 
Primary Education, Secondary Education, Teacher Education, Technical and Vocational 
Education, Non-formal Education and Curriculum Development and Evaluation. Each 
department is supervised by a Chief Education Officer; the Planning Unit is headed by a 
Principal Planning Officer, and all Planning Unit staff are employees of the Ministry of Finance 
and Development Planning; the Unified Teaching Service is headed by a Director and the 
Bursaries Unit by a Secretary. The Brigades, an organization that provides technical training 
for junior-secondary school leavers and drop-outs relates directly to the Permanent Secretary. 
The University of Botswana is an autonomous institution, controlled by a Council but maintained 
by public funds from the Ministry of Education. Students receive government bursaries to attend 
the university and are then are bonded for a limited period after graduation to work in the public 
sector. 

The day-to-to day management of primary schools is undertaken by District and Town Councils 
under the overall direction of the Ministry of Local Government and Lands. Since the 
abolishment of secondary school fees in 1988, all public education in Botswana is free. 
Secondary schools are divided into three categories: government, mission-aided and community 
schools. Community schools are controlled by local Boards of Governors. The Ministry has 
a policy assuring access for disabled or handicapped students, and a special unit within the MOE 
is set up to liaise with other agencies that provide assistance to handicapped students. 
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EDUCATIONAL POLICY AND PLANNING 

Botswana educational policy is mainly determined through National Development Plans which 
are prepared and implemented in six year periods. The Seventh National Development Plan 
began in April 1991. The Ministry of Education (MOE) is the govemment organization 
responsible for determining, coordinating and implementing educational policy. Serving as the 
bluq^rint for Govemment of Botswana policies, the National Development Plan preparation 
process involves an extensive cycle of developmrat, review and revision. These policies are 
developed and implemented through the Ministry of Education's Policy Advisory Committee 
(PAC), whose membership consists of heads of departments and units of the Ministry. The 
committee is chaired by the Minister and, in his 2d)sence, by the Permanent Secretary. If 
decisions made at this forum are of a sorious nature (particularly if they have financial 
implications), they go to Cabinet for s^proval. This committee meets at least four times a year. 
Any head of dej^utment wishing to bring a policy issue to the PAC for consideration must 
prepare a paper on the issue, outlining problems and providing policy proposals. The psptr is 
circulated to other members, and issues are discussed at a PAC meeting. Some decisions, such 
as housing policy for teachers, membership of school boards and transfers of teachers are 
referred to Cabinet. 



INFORMATION FOR DECISION-MAKING 

The Botswana Ministry of Education has long understood the importance of accurate and timely 
mformation for decision making. Since independence, the Central Statistics Office (CSO) of the 
Ministry of Finance and Development Planning (MFDP) has produced an annual Educational 
Statistics publication, which contains data on schools, teachers and students. This information 
is widely used for MOE planning and programming decisions. However, as the educational 
system in Botswana expanded, the time required to collect and analyze data also increased. The 
CSO publication is usually released about two years after the completion of the school year, 
limiting its usefulness for many planmng purposes. In addition, many of the departments need 
much more detailed information than can be provided by the CSO. Consequently, several of 
the departments within the Ministry of Education have begun to carry out their own data 
collection. 

During the past few years, the Improving the Efficiency of Educational Systems (TEES) Project 
has assisted several MOE departments in establishing computer databases, including: the 
Planning Unit, Unified Teaching Service (UTS), Department of Primary Education, Department 
of Secondary Education, Department of Non-formal Education, and Department of Bursaries. 
With the rs^id expansion of information systems, the MOE has had problems with both 
duplication of effort and inconsistencies among data sources, due in part, to variations in the data 
needs of individual departments and the collection of data over different periods of time. For 
example, both the Primary Department and the Unified Teaching Service collect information 
about teachers. While the UTS only tracks teachers on the UTS payroll, the Primary 
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Department collects data on students and teachers in all registered primary schools, including 
mission and private schools. This information is needed to calcuUite the number of eligible 
school-age children enrolled, as well as those who will need to be accommodated in junior 
secondary schools* As a result of these differing requirements, two estimates of the teacher 
population exist. This is not a problem as long as the parameters of the two sets of data are 
clearly stated. However, in the past, confusion has occurred when different figures were quoted 
in various rq)orts and speeches. 

As the number of databases grew, the need for information coordination became more evident. 
Consequently, a Database Managers Group was established to exchange information, discuss 
issues related to the development of individual databases and to establish common procedures 
for use by all MOE departments/units regarding coding, selection of software, and disseminaticHi 
of information. Meeting once a month, this group has operated effectively for almost five years. 
Membership consists of individuals who are responsible for developing and maintaining 
databases in each MOE dq>artment, as well representatives from the Central Statistics Office and 
the University of Botswana. 

While establishment of the Database Managers Group helped address many data problems, it 
soon became clear that a central source of information comprised from the various databases was 
also needed. Several alternative approaches to developing an Educational Management 
Information System (EMIS) were considered. It was decided that efficiency analysis would be 
the most useful approach. Efficiency analysis, according to Windham (1988), is a framework 
for decision-making that assumes scarcity of resources and the need for difficult choices. There 
are several reasons for selecting an approach based on efficiency analysis. Botswana is in the 
process of implementing policy reforms that will have far-reaching impact on the educational 
system. The Government has set a goal of achieving universal access to basic education, an aim 
which requires a major expansion program at the junior secondary level. At the same time the 
Government is also committed to improving the quality of education. 

Careful planning ensured sufficient resources for achieving the objectives for the initial 
expansion period. The number of private and community junior secondary schools (QSSs) grew 
from 42 in 1985 to 146 m 1992, and the percentage of primary students admitted to junior 
secondary school increased from 36% to 81 % . However, this expansion has proved to be more 
costly than originally anticipated, and there is concern about the cost of sustaining this rate of 
increase. Furthermore, a possible decline in the growth in Botswana' s economy could eventually 
diminish the resources available to the educaticHi sector. The interest in quality improvement 
and cost containment have resulted in a need to seek ways of managing system growth and 
quality improvement in a cost-effective way. At the same time it has become necessary to begin 
monitoring how well the changes address specific policy objectives. Educational systems need 
to be efficient both in generating and in expending resources^ and these two objectives are not 
always perfectly compatible. 

An Educational Management Information System specifically designed to facilitate efficiency 
analysis is an important first step toward meeting these needs. The elements of the EMIS 
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described here provide the framework to analyze efficiency. This system will also assist in 
determining the extent to which the Ministry is accomplishing the objectives outlined in the 
National Development Plan and help to coordinate information from the various Departments 
within the MOE. 



A FRAMEWORK FOR TRACKING AND ANALYZING 
EDUCATIONAL EFFICIENCY 



With this in mind, a framework for tracking and analyzing trends related to educational 
efficiency in Botswana was developed. In order to establish this framework, seven basic steps 
(outlined by Easton, Holmes and Williams, 1990) were followed: 

1. Development of conceptual definitions for principal terms and concepts to be used; 

2. Articulation of the underlying model relating the indicators to each other; 

3. Selection of indicators; 

4. Data collection; 

5. Data analysis; 

6. Consideration of other technical and methodological methods; 

7. Consideration of other organizational and instructional issues. 
CONCEPTUAL DEFINITIONS 

In developing the Botswana EMIS model four key terms: indicator, educational efficiency, 
educational quality, and equity were defined. There are many definitions of these terms. The 
boxes which follow depict but a few of these. 
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INDICATOR 



^An fndfcfttor ti^ • 
st«ttstfc About th9 
cducfttiocmi %ittm ttiiit 
r«v«ft(» soMithintf about 
1 1» p«rf Dnmrncifc or 

nt «!thtr i*Mttt« Of It 
r*l«t«d tof t ditirod 
: outi:cw« of tht: tdomfontt^ 
sytt«i or dttcf ibtt; cort: 
f«fttur» of that «yttoii,>» 
{^ith^ 1988) 



pOfMift fWMrd1at«E««or 
naarty tmtdf at***^ 
infaronctt' . itwut. ; th* 
ptrfomanct ••.••.of ^ tho 
; (odkicatlonill vft^mJ^- 
CCobba, 1989>. 

thtyr <*prdvfdi iwiasMroa of 
^Houa cmpijnmtt of thfe 
odueatf orail ' n^tm^ «t 
util oa InfofMstfort about 
hoii thtta coNponanta work. 
to9athar to provfda tht 
conditfcih^ of tha. aysta« 
and ^ ^mvaHi'^A^m/ tha 
tOndftfon of the aystM 
Gvar tfiia.« (Oddart, t990> 



: *t ndf cat or*::- V irrr -^^^ proxias: 
uted to rapreatnt the 
Jurtdartyf n9 raaiity* of i^tha^ 
syatanr or poanai. • « and 
ara nacasaanly an ovar ^ 
aiwptification . of thia 
raaUty.* fChapniwi,. t990> 



."^I«dfcatora:: . do:: ^not 
axptaf :: thay : : only 
point." <Bottani, 1990) 



EDUCATIONAL EFFICIENCY 



The concept of efficiency 
provides a broad perspective 
from which to analyze an 
educational system; one in 
which the costs of educational 
inputs and processes can be 
related to benefits, such as 
improved effectiveness. As 
noted by Easton, et. al. 
(1990:11), this concept has 



m aanarai . ttrmai 
■ : »af f f ef ancy-:-:'raf art': rto: ■ tha- 
atata of baint product fva^ 
capofata, and coii^atant; 
But in tha tachnicat 
aanaa;:;- i t= ■rtfara:.: to- = tha- 
: raiatlomfthip batuaanycoat: 
and af faetivanaaa> 
(Hartualt^ 1989) 



»(Eff<daneyl rafara to 
tha andaavor to get tha. 
lioat.: output firott flvan. 
tKiahtltiaa of' . Inputa; 
ihuav^ «aaauraa: of 
.BffMwcY ;tdaatiy ara 
^iuahtltativa ratfoa^ of 
output to. input aoiia 
fofHk*^ (Cobba, 199»> 



meaning only if ou^uts and 
outcomes are correctly 
specified and measured. 
However, outputs and 
outcomes may vary 
significandy from one country 
or region to another. These 
authors also point out that ""they 

may involve affective, as well as cognitive results, group, as well as individual effects, and 
distributional, as well as summational considerations.** 



••Ef f 1 CI ancy:; i a :: , a ratio/ 
not an absolute Mgnf tuda^: 
(£aaton> at. al.V 1990): 

■ ''.'-'*«*«*******•***'' - 

^ . v^i lntamaV : tducatf ona I : 
afficiancyi;;axista uhera 
tha valua of aducattonai 
output . fs Maxinlzed for a 
aWan: coat of Irfiuta: (or 
Mhara coat is : Jifniwizad 
for a apacif fad : value: of 
output>w»^ .fttyrmtt- 
efficftncy^ : rafara the 
ratatlbrishfp batMeanx^^^^ 
aducatf ont i : ayiteai and the : 
■axtarnai::.-.;:.. "anvil ronwanr 
(l.av^ the world Of uork 
and::laisura);x ChapMirt and 
UindhaM, 1986) . 
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EDUCATIONAL QUALITY 



According to Snyder and 
Nagel, (1988:14), quality of 
education is probably more 
related to the content and 
process of education than to the 
extent of schooling. They 
maintain that indicators of 
quality and efficiency '•must be 
multifaceted and probably 
highly inferential in order to 
capture the richness and 
complexity of schooling/ 



^■=i«^::S;dtf^iniifi^^ 

itf proof of / i^dtU^e. 
ThtH^ If «U tiMdilnt In 

school: h«v*: ; : = N«tt«f<# 

. <teort«t« th« 9r<KJ|> utli b# 
coniit<krod * iliigiV qu«lf ty 
»t»ff* On tNr othtf: h«ncl; 
quoHty it dtfffMd:«s tht 
■ lirov«n>/«bi t ! ty;--: W:"|rodue«^'^^^ 
' rttults. : .. <f «ttofv, 

1989) • . . 



ovftittitfon.: Although It 
«iy fo* . Inf lutneod.:.: by 

■"phyiic%i- .:\cond1t1on«?:^«nd-'^ 
circgnmtancttj; c^ilty 
•n t Hi I s f t f t i Ptf > 

' «tt{tUGtev:;.ttxi-:-'V«tuesv--::'ancl> 
ft {s lioreithin'tht stM Of : 
objective rr>dft*tOf*;;* 
(Snyder, t990) 



EQUITY 



^y eqMity ifi education^ ue ^nean . fairness 
i betiMirtVrdisttnouishebte groups in terw of 
access to; participation In; and achievesMnt of 
the educatfbnat systesi.** (Cobbe^ 1990) 



Cobbe (1990) contends that to measure 
equity, we must have data that distinguish 
between the groups among which we wish 
to ensure equitable distribution of resources 
or services. Possible characteristics for 
analysis include gender, geographic location 
and private/public education. Examples of 

indicators of equity are statistics such as net enrollment ratios, cycle completion, number of 
textbooks available to students and the extent of teacher training. 



ARTICULATION OF THE I3NDERLYING MODEL 

The model selected as the basis for the EMIS in Botswana is a variation of the Context, 
Input, Process, Product (CIPP) Model, which has been widely used by educational evaluators 
since the 1970* s. This model classifies educational mformation into the categories identified 
by Stufflebeam as Context, Input, Process and Product. Windham's (1988) discussion of 
educational efficiency uses a similar taxonomy but classifies information into categories of: 
Inputs, Process, Output and Outcomes. The Botswana model incorporates components of 
both models to include the categories described below. 
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Context* Context information describes the current conditions, issues, opportunities and 
constraints in the environment. It is a form of needs assessment which assists in identifying 
the types of programs appropriate to a given setting. 

Inputs. Inputs are the resources available for educational production. Inputs include 
characteristics of students, schools, teachers and instructional materials. In examining inputs 
it is necessary to analyze recurrent unit costs and development costs. 

Process. The process stage of educational production refers to the means or processes by 
which educational inputs are transformed into educational outputs. According to HartweU 
(1989:4), "Processes are what happens to inputs within the school setting (or non-formal) 
setting and involves management, leadership, the acts of teaching, the process of learning, 
inspection, school activities, supervision of teachers, and the operations of the Ministry of 
Education in organizing and utilizmg its resources and fiances.** A specific process for 
promoting educational ou^uts is sometimes referred to as an educational technology. 

Outputs. The direct and immediate effects of the educational process are the outputs. They 
include cognitive achievem^t, manual skill development, attitudinal changes, and behavioral 
changes. Outputs can be divided into two categories: educational attainment and educational 
achievement. The former is quantitative and includes the number of students attain a given 
level of education over a given period of time. The later reflects social values, policies and 
educational aims. It includes values, attitudes, competencies and knowledge that students 
achieve as a result of their educational experience. To assess educational effectiveness, we 
should know not only what students achieved in relation to curriculum objectives, but how 
the school and educational experience assisted them to gain knowledge, skills and 
competencies. 

Outcomes. Educational outcomes are the less direct and less immediate results provided 
through the interaction of educational outputs with the social environment. Outcomes cover 
a wider range of individuals than just the leavers of the educational system, a longer time 
period, and a larger political-geographical reference area. Indicators of outcomes are the 
levels and nature of productive work and employment of school leavers and graduates and 
their wages or income. Outcomes also include the contribution of educated individuals 
within a family and community. 

SELECTION OF INDICATORS 

A preliminary set of indicators of educational efficiency in Botswana was compiled with 
assistance from the lEES Resident Technical Adviser (RTA) and Florida State University 
staff. The Botswana Data Managers Group and the TEES RTA then participated in a series 
of working sessions to identify items that are currently available, as well as those that are 
needed but are not being provided by any of the departments. 

It was decided by the Data Managers Group that the initial database should only include 
information currently available, rather than data requiring additional research. Furthermore, 
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in Older for the database to be sustainable over time, it was concluded that it should be 
manageable in size. If first efforts are too ambitious, participants are likely to be overwhelmed 
by the enormity of the task of collecting and maintaining the data. The objective is not to 
duplicate databases being maintained in other departments but rather to select key summary 
information needed for cmtral decisions. The system is designed to cross-referrace other 
databases and recent research so that managers can quickly discern where to go for detailed 
information. Based on these criteria, a set of indicators was selected for inclusion in the system, 
as well as those which should be added at a later date. 

Tim Goddard, a former Peace Corps Volunteer who helped to set up the Primary Dq)artment 
database, was hired as a consultant to write the programs for a menu-driven EMIS. A set of 
data entry forms was then disseminated to each member of the Database Managers Group and 
meetings were hdd with individuals to clarify data items and ensure consistency in reporting. 
The database was organized into categories (by department), as shown in Figure 1. This 
diagram is a modified version of a flow-chart compiled by Easton, et. al. (1990). Changes to 
the original model were based on the Database Managers Group's assessment of what 
information is both necessary for planning and decision-making and realistically attainable. The 
boxes with broken lines represent categories of information that are not currently available but 
should be added at a later date. Acomplete list of data items, by category, is presented in 
Appendix A. You will note that the majority of information Ms into the Inputt Process, or 
Output category. Some Context information, such as per capita incomes, average life 
expectancy, and other socio-economic indicators will be added at a later time. Information on 
Outcomes is much more difficxilt to obtain and will require additional research. 

Appendix B depicts the EMIS Main Menu and the first set of screens associated with Primary 
Department information. Similar screens appear for each of the departments listed on the Main 
Menu. Altogether, there are over 100 screens which can also be printed. As previously noted, 
the EMIS in Botswana is designed so that data sets can be expanded as more information 
becomes available. Some data items that were considered to be important but which were not 
currently available were built into the system. 

DATA COLLECTION AND ANALYSIS 

Data collection for the first year was carried out over a seven month period from August to 
February, 1992. This process took much longer than originally anticipated, largely because of 
competing demands on Ministry of Education staffs time. In almost all MCE departments, 
maintainmg databases represents only one of many duties of the data mans^er. Also, data 
managers' estimates of what data could be readily obtained were overly optimistic. Much of the 
information was either unavailable or widely scattered. For example, information from the 
newly formed Teacher Education Department took six months to collect and required that the 
representative to the Data Manager's Group travel to teacher training institutions and education 
centers in various locations around the country. This had to be accomplished while he was 
juggling many other activities, such as developing materials and conducting teacher training 
workshops. However, he felt that this was an extremely useful exercise that helped him to 
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formulate databases for the department's own use. Now that the teacher training college 
principals and the education center directors are aware that they will be expected to provide this 
information on an annual basis, data collection should flow much more smoothly. Data 
managers in other departments have reported similar experiences. 

TECHNICAL AND METHODOLOGICAL ISSUES 

Consistency in Time Period Covered 

In order to allow comparisons, it is important that the time covered in the statistics provided by 
all departments be consistent. Although efforts were made to ensure that all information covered 
the same time period, this was not always possible for two reasons. First, the length of time 
it took for each department to collect the data ranged from one to seven months. Even though 
the time frame for which the data were to cover was explicitly stated, several departments 
provided the most up to date figures available, rather than figures from the specified time firame. 
Second, there is considerable variation in the timmg of school calendars at different levels in the 
educational system in Botswana. Primary, junior and senior secondary schools and the teacher 
training colleges operate on a trimester system, with classes running from January to April, from 
May to August, and from September to December. The university and the vocational training 
centers operate on a two-semester system, with the first semester beginning in August and 
ending in December and the second semester operating from January to April. These differing 
calendars make it extremely difficult to adhere to a consistent time fiame in all the databases. 



Equity Issues 

One of the objectives of the EMIS in Botswana is to collect information relating to equity issues. 
Although it was possible to disagregate some information into gender and district-level analyses, 
additional mformation is still needed if the EMIS system is to be effective in assessing 
educational efficiency. 

National vs. School-Level Data 

Snyder and Nagel (1988) maintain that one of the major short-comings of most management 
information systems is the emphasis on indicators of educational expansion (e.g., number of 
secondary school graduates, percent of school-aged population enrolled in school) at the 
national-level at the expense of school-level data. Although such information allows 
comparisons to be made among cases and over time, thus allowing time-series analysis or 
inter-country comparisons, it does necessarily indicate a more educated population. They argue 
that quality cannot be measured in terms of quantity (or expansion); more is not necessarily 
better. National-level measures of achievement (such as number of schools, teachers, and 
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literacy rates) also pennit comparisons across cases and time but share the same validity 
problems as measures of educational expansion. National-level data indicate the extent of 
schooling but are not helpful in assessing the content or process of schooling. For example, 
information about the number of textbooks available or die existence of a national curriculum 
does not tell us how many teachers are actually usmg the curriculum nor the methods they are 
employing to teach the materials. Hence, these authors believe that the most useful unit for the 
focus of data collection and analysis is the school. 

What is Missing? 

In developing the EMIS system for Botswana we have attempted to address some of the issues 
described above by including information about recent studies which are concerned with issues 
of educational quality at the school level. Howev^, currently, most of the data available in 
Botswana are quantitative, national4evel information. There is a preponderance of Input data, 
primarily related to the existence of resources (schools, teachers, students, facilities) but very 
little information about the quality of school management, teachers' instruction or curriculum 
quality, factors that are essential to measuring the efficiency of an educational system. 

The database does not currently include information on Community Participation nor on School 
Management (see Figure 1). However, references to research carried out on these topics, such 
as USAID-funded headmaster and teacher incentives research and classroom observation studies, 
will be incorporated. Additionally, new research results will be added to the database as they 
become available. 

Dr. Kent Noel, former Chief-of-Party for the USAID-fimded Botswana Junior Secondary 
Educational Improvement Project (1992) argues that while national-level information is useful, 
we need to be concerned about the quality as well as the quantity of educational products. He 
contends that monitoring of quality should take place not just at the national and school level but 
at all points in between. Noel believes that the lack of information in the Botswana EMIS on 
the curriculum development and implementation process is a serious omission. He states that 
there is a need to pay more attention to factors that affect curriculum quality and efficiency, 
particularly those relating to the design, development, implementation and evaluation of 
curriculum materials. He goes on to assert that the EMIS needs to include data from all 
categories of information (Context, Input, Process, Output, Outcome). This would assist policy 
makers in making decisions concerning 

how time, money and other resources could best be spent in improving curriculum quality. The 
database should be modified to include such information. 

Another critical area that was omitted from the preliminary version of the system is 
university-level information. The decision to exclude university data from the first year's data 
collection was made purely on the basis of the limited time avaUable to write the programs and 
collect the data. These data should be added to the system as soon as possible. 
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A major gzp in existing information also relates to cost data. Although budget information is 
relatively easy to obtain, very little cost data is currently available. This is particularly true 
for primary education, where school budgets are decentralized and are the responsibility of the 
Ministry of Local Govemment and Lands. All teachen employed by the Unified Teaching 
Service are paid by the Ministry of Finance and Development Planning. Despite the fact that 
these records are computerized, it has not been possible to obtain a breakdown of teacher's 
salaries by level (primary, junior secondary, and senior secondary). Consequently, the cost 
analyses carried out by the Planning Unit are frequently based on estimates, rather than actual 
figures. Similarly, cost analyses have not been carried out for many of the activities outlined 
m the National Development Plan (NDP). For example, the education section of NDP 7 states 
that during the next plan period, the Ministry of Education will begin the process of localizing 
the Cambridge (Secondary School Leaving) Exam. Activities are to include training of 
examiners and markers, writing and adapting syllabi, procuring equipment and planning physical 
facility requirements. However, no cost analyses have been conducted to determine initial and 
recurrent costs of implementing these activities. Cost data of this nature are imperative to 
accurately assess system efficiency. 
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Disseminatioii of Inf onnation 



Data will be made available to all heads of Departments, central MOE officials, and donor 
agencies in the form of an annual report. Dissemination should not be limited to the central 
level of administration but should include the school and district level as well. Copies of the 
annual rqport will be provided to Field Education Officers, District Education Officers, 
Headmasters/ Headmistresses, Teacher Training College Principals and Education Centre 
Directors. The Primary and Secondary Departments have found that disseminating statistical 
information to the school and district level has resulted in increased cooperation in data gathering 
efforts and contributed toward better management and record-keeping practices by those required 
to provide data. 

Sustainability of the Informatioii System 

As noted earlier, most of the data contained in the EMIS are derived from existing MOE 
databases. Although this has substantially facilitated the data collection process, the continuation 
of the ENQS is, to a large extent, contingent upon the continued operation of these systems. 
However, several of the systems are presentiy faltering due to a shortiige of trained staff and the 
high rate of staff turn-over. Individuals responsible for operating the Primary and Technical 
Education Department databases have left the Ministry and have not been replaced. 
Additionally, database managers for every existing database within the MOE have reported 
problems in finding sufficient staff to enter and maintain data. Similarly, the person in the 
Planning Unit responsible for operating and updating the centralized EMIS has many other 
responsibilities, with littie time for maintaining the system. These problem must be resolved if 
the system is to be sustained over time. 



Lessons Learned 

In the process of designing and implementing an Educational Management Information System 
for Botswana, several procedural/organizational issues have emerged. In addressing these issues 
a set of basic operational guidelines was developed. These are described below: 

1. Decisions regarding the content of the EMIS must be made in consultation witii parties 
from all departments expected to provide data. Most department heads and data 
managers are regularly bombarded with requests for information. If they do not consider 
the data to be important, they will not be willing to take time away from other tasks to 
collect it. In deciding what to include, one must carefully consider the questions: 1) who 
will use the mformation; 2) what will it reveal about the efficiency of the educational 
system; and 3) how will the mformation be used? 
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2. It is important that a permanent member of the Ministry staff be assigned responsibility 
for collecting and maintaining the system. In addition, other staff members need to be 
assigned to assist this individual. Because of high staff turn-over, it is imperative that 
more than one person be trained in maintaining the system. Retaining staff trained in 
data entry and analysis continues to be a problem for all departments in the MOE, as 
well as other Ministries. Once individuals are trained in using computers, they are often 
able to double or triple their salaries by moving into the private sector. Incentives must 
be found to retain trained staff. 

3. A users' manual must be provided, containing complete system documentation and clear 
and detailed written instructions for operating and updating the system. Instructions must 
be easily undwstood by someone who has never operated a system of this type. 

4. The system must be flexible enough to allow for modifications as more data needs change 
and more information becomes available. 

5. The system must have support from top administration. Data collection is time 
consuming for those responsible for updating the information system, as well as for those 
who must provide the data. If the system is to be sustained over time, both the data 
collectors and the data providers must perceive that top-level administrators value the 
information. 

6. Because each department maintains its own database, department heads and database 
managers must have an interest in receiving information about other departments. 
Consequently, it is crucial that the EMIS be demonstrated and its potential benefits be 
explained to those in key decision-making positions, to department heads and to those 
responsible for providing the data. Furthermore, the information must be both timely 
and accurate. 

Next Steps 

As previously noted, this information system is a first step toward addressing data problems 
within the Ministry of Education. It should be viewed as a starting point rather than a completed 
system. It is important that additional data be gathered to fill information gaps and that the 
system be to modified to accommodate changing information needs. Further, if the system is 
to survive, staffing problems must be resolved. Fortunately, a recent decision by the 
Government of Botswana to create an Education Planning and Statistics Division within the 
Ministry of Education represents significant progress toward this end. Presently, Planning Unit 
staff must divide their time among many competing demands, with information collection and 
database management ranking low in priority. The creation of the new division will assist in 
alleviating this situation, with the division consisting of five units: 

• Education Projects, Monitoring and Evaluation; 

• Educational Planning; 
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• Educational Infonnation and Statistics; 

• Educati(Hial Research; and 

• Division Management. 



This arrangement will provide staff specifically responsible for the collection and maintenance 
of educational information. In addition, a research component will enable the system to be 
expanded to include data which are not cunenUy available. Having an EMIS system already in 
place will make the initial phase of organizing the work of the Information and Statistics 
component considerably easier. One of the fint tasks faced by this unit will be to work with 
the individual MOE departments to ensure that the departmental databases are sustained. 

The development of a systematic means of tracking and coordinating data within the Ministry 
of Eduction has been a long and arduous process. This system, which has evolved over several 
years, is a culmination of the work of many individuals. The MOE is now in a position to reap 
the benefits of this work. Through the systematic collection, analysis and dissemination of 
information about the educational system, die Ministry will be better able to assess the success 
and usefulness of its educational programs. The availability of such information to decision 
makers should lead to more effective polic/ and a more efficientiy operating educational system. 
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APPENDIX B 



Box 1 depicts 
the EMIS Main 
Menu. Data are 
stored by 
Department/ 
Functional area. 
Each of the 
departmental 
databases listed 
on the Main 
Menu contain 
data from the 
six categories 
illustrated in 
Box 2. 



BOTSWANA MINISTRY OF EDUCATION 
EFFICIENCY INDICATORS 
MAIN MENU 



Primary Indicators 
Junior Secondary indicators 
Senior Secondary Indicators 
Teacher Education Indicators 
Technical Education Indicators 
Non-Formal Education Indicators 
Brigades Indicators 
Bursaries Indicators 
UTS Indicators 
Curriculum Dev. Indicators 
Headquarters Indicators 
Data References 
Create System Backup 
Data Update 
Exit 



1 


-> 


2 


-> 


3 


--> 


4 


--> 


5 


--> 


6 


--> 


7 


-> 


8 


--> 


9 


--> 


10 


--> 


11 


--> 


12 


--> 


13 


--> 


14 


--> 


15 


--> 



ENTER SELECTION— > 



Box 1 Main Menu 
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The menu 
shown in Box 2 
(Primary 
Education 
Menu) appears 
on the screen 
after selecting 
Option 1 
(Primary 
Indicators) on 
the Main Menu. 



PRIMARY EDUCATION 
EFFICIENCY INDICATOR CATEGORIES 



2 



Resource Management 



Student Information 



3 


-> 


Teacher Information 


4 


-> 


Educational Quality 


5 


-> 


Equal Opportunity 


6 


--> 


Financial Management 


7 


-> 


Data Entry Form 


8 


-> 


Exit 



ENTER SELECTION-- > 



Box 2 Primary Education Menu 
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1.0 PRIHARY EDUCATION 
l.l.l RBSOURCS MANAGEMENT (Screen 1) 



COMPARISO N YEAR » 


1989 




MOST RECENT YEAR'* « 


1991 


H No* primary School 


584 




No. primary schools 


654 


1 Total enrolinent 
1 


275,437 




Total enrolment 


308,840 


1 Council-owned schools 


540 




Council*owned schools 


582 


1 Mission-owned schools 


17 




Mission-owned schools 


11 


1 Private-owned schools 


27 




Private*owned schools 


61 


I Adult/Evening schools 


N.A. 




Adult/Evening schools 


15 










. n . A. 


Schools with Special Ed 


N.A. 






Total classrooms 


5,775 




Total classrooms 


6,627 


Total streams 


8,218 




Total streams 


8,627 


Total teachers 


8,529 




Total teachers 


9,709 




n • A* 




lo^ax ceacner nouses 


2, oj4 


Total books issued 


N.A. 




Total books issued 


N.A. 


Total Dept. Officers 


N.A. 




Total Dept. Officers 


41 


Box 3 




Box 4 





Box 3 Notes: 



Source: Central Statistics Office, 1989. 



Box 4 Notes: Source: Primary Department, 



1.0 PRIMARY EDUCATION 
1.1.2 RESOURCE MANAGEMENT (Screen 2) 



COMPARISON YEAR'' » 1989 



National Aver age Ratios 
Students per school 471:1 
Students per teacher 32:1 
Rooms per school 9:1 
Streams per school 14:1 
Houses per school N.A. 
Issued books per student N.A. 
Schools per officer N.A. 



MOST RECENT YEAR" = 1991 



National Average Ratios 
Students per school 472:1 
Students per teacher 
RoomL per school 
Streams per school 
Houses per school 
Issued books per student N.A 
Schools per officer 



32:1 
10:1 
13:1 
4:1 



15:1 



Box 5 



Box 6 



Box 5 Notes: 
Box 6 Notes: 



Source: Central Statistics Office, 1989. 
Source: Primary Department. 
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1.0 PRIMARY EDUCATION 
1*2 STUDENT MANAGEMENT 



COMPARISON YEAR" « 1989 



Total Enrolment'* 275,437 

Standard 1 46,277 17% 

Standard 7 37,031 13% 

Total males 133,608 49% 

Total females 141,529 51% 

Special Ed. % 

Total dropouts^ 1,765 1% 

Total repeaters^l3,594 5% 

Est. primary age pop.^ 227,984 

Net enrollment ratio" 92.5% 

Gross enrollment ratio 121% 



Box 7 



MOST RECENT YEAR" « 1991 



Total enrolment'* 


308,840 


Standard 1 51,235 
Standard 7 39,938 


16% 
12% 


Total males 150,205 
Total females 158,635 
Special Ed. 503 


49% 
51% 
0% 


Total dropouts" 2,865 
Total Repeaters'*" 14,154 


0% 

5% 


Est. primary age pop.'* 


270,322 


Net enrollment ratio" 
Gross enrollment ratio 


114% 



Box 8 



Box 7 Notes: 



" Source: Central Statistics Office, 1989 (except where noted) 
'* Includes repeaters. 



" As % of current year total. 
^ Source: CSO, Population Projections i 
" Includes students age 7-13. 



1981''2000, 1987. 



Box 8 Notes: 



" Source: Primary Department (except where noted). 
^ Includes repeaters. 
" As % of current year total. 
^ Source: CSO, Population Projections: 
" Includes students age 7-13. 



ISai-ZOOO, 1987. 
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1.0 PRIMARY EDUCATION 
1*3 TEACHER MANAGEMENT 



COMPARISON YBAR^ » 1989 



Trained male tchrs. 1,537 18 
% 

Trained female tchrs 5,675 66 
% 

Untrained male tchrs 210 2 
% 

Untrained female tchrs. 1,107 13 
% 

Total no* of teachers 8,529 
Percent headteachers male N.A 
Inrcent headteachers female N.A 



Average age all teachers 



N.A 



Total no. expatriates 248 2% 



MOST RECENT YEAR'* «1991 



Trained male tchrs. 1,779 18% 
Trained female tchrs. 6,866 71% 
Untrained male tchrs. 195 2% 
Untrained female tchrs. 868 9% 
Total no. of teachers 9,708 

Percent headteachers male 56% 
Percent headteachers female 43% 

Average age all teachers 40 

Total no. expatriates 53 0% 



Annual new hires 
Annual terminations 
Annual transfers 



1,313 13% 
21 0% 
475 5% 



Box 9 



Box 10 



Box 9 Notes: Sources Central Statistics Office, 1989. 

Box 10 Notes; ^ For % trained/untrained, male/ female teachers & total teachers 
source is Primary Department; (figures include all teachers); 
all other figures are from UTS. 
* Figures for UTS-employed teachers only (from UTS database) are: 
Total no. primary teachers » 9,575 

Trained male teachers « 1,603 17% 

Trained female teachers^ 6,467 68% 
Untrained male teachers- 239 2% 

Untrained female teachers* 1,266 13% 
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1*0 PRIMARY EDUCATION 
1*4 QUALITY INDICATORS 



COMPARISON YEAR » 1989 



MOST RECENT YEAR « 1991 



Standard 7 enrolment^' 



37,031 



No. sitting for PSLE* 38,073 
% of PSLE grades above D 75% 



% of std. 7 students 
continuing to Form 1^ 



Std. 1 to Std. 7 
progression rate^ 

Std* 1 to Form 1 
progression rate^ 



60% 



% of students who sat for 

PSLE admitted to Form 1^ 60% 



91% 



69% 



Teacher training workshops N.A. 

Total workshop attendance N.A. 
% of teachers attending 

workshops during this year N.A. 

No. of school inspections N.A. 



Standard 7 enrolment'' 



39,938 



No. sitting for PSLE* 38,223 
% of PSLE grades above D 72% 



81% 
84% 
89% 



% of Std. 7 students 
continuing to Fom l'* 

% of students who sat for 
PSLE admitted to Form 1^' 

Std. 1 to Std. 7 
progression rate'* 

Std. 1 to Form 1 

prgression rate''^ 85% 

Teacher training workshops N.A. 

Total workshop attendance N.A. 
% of teachers attending 
workshops during this year N.A. 

No. of school inspections N.A. 



Box 11 



Box 12 



Box 11 Notes: " Source: Central Statistics Office (CSO) , 1989. 

* Standard 7 students sitting for Primary School Leaving Exam 
(PSLE) in 1989; Source: CSO, 1989. 

^ Percent of 1989 Std. 7 students entering Form 1 in 1990 (gross 
progression rate, including repeaters); Source: CSO, 1989; 
1990) . 

* Percent of students who sat for PSLE who were admitted to Form 
1 in 1990. 

* Std. 1 (new entrant) students beginning in 1983 and 
progressing to std. 7 in 1989; gross progression rate, including 
repeaters is 114%; Source: CSO, 1990. 

Std. 1 (new entrant) students beginning in 1983 and 
progressing to Form 1 in 1990; gross progression rate, including 
repeaters is 69%; Source: CSO, 1990. 

Box 12 Notes: ^ Source: Primary Department. 

* Std. 7 students sitting for PSLE in 1991; Source: Secondary 
Department . 

^ Percent of 1991 std. 7 students entering Form 1 in 1992; 

(gross progression rate, including repeaters); Sources: std. 
7 enrolment from Primary Department; Form 1 enrolment figures 
from Secondary Department. 

Percent of students who sat for PSLE who were admitted to Form 
1 in 1992. 

* Std. 1 (new entrant) students beginning in 1985 and 
progressing to Standard 7 in 1991; gross progression rate is 
104%; Sources: CSO, 1990; Primary Department. 

Std. 1 (new entrant) students beginning in 1985 and 
progressing to Form 1 in 1992; gross progression rate is 
84%; Sources: CSO, 1990; Primary Department. 
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1.0 PRIMARY EDUCATION 
1.5 INDICATORS OF EQUAL OPPORTUNITY 
MOST RECENT YEAR'* «1991 



r 



DISTRICT 


ROLL 


%BOYS 


%GIRLS 


STUD : TCH STUD : ROOM 


NORTH EAST 


12,393 


51 


49 


29 J 


1 40 : 




CENTRAL NORTH 


25,965 


50 


49 


31 : 


1 48 : 




CENTRAL CENTRAL 


54,982 


49 


50 


31 : 


1 49 : 




CENTRAL SOUTH 


25,698 


49 


50 


33 J 


1 52 J 




K6ATLEN6 


14,344 


49 


53 


30 J 


1 42 J 




KWENEN6 


39,065 


46 


51 


32 : 


1 48 : 




SOUTHERN 


38,764 


48 


51 


31 \ 


; 1 46 : 




SOUTH EAST 


8,343 


48 


51 


30 


: 1 38 




KHALA6ADI 


8,967 


48 


51 


26 : 


: 1 36 - 




6HANZI 


6,681 


49 


50 


29 


: 1 37 J 




NORTH WEST 


25,261 


48 


51 


30 : 


; 1 49 




GABORONE 


20,655 


48 


51 


33 


: 1 46 - 




FRANCISTOWN 


10,781 


48 


51 


33 


: 1 43 




LOBATSB 


6,010 


46 


53 


36 


: 1 49 




SELEBI-PHIKWE 


8,534 


47 


52 


36 


: 1 53 




JWANEN6 


2,397 


48 


51 


31 


: 1 50 





Box 13 



Box 13 Notes: ^ Source: Primary Department 
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1.0 PRIMARY EDUCATION INDICATORS 
1.6 FINANCIAL INDICATORS 



COMPARISON YEAR'^ » 1989 



Total MOS budget P 220,480,660 

% for Primary Dapt. 0% 

Total Dept. Budget'^* P 1,850,140 

% for sub head 701 75% 

% for sub head 702 7% 

% for sub head 703 1% 

% for sub head 704 5% 

% for sub head 706 6 707 2% 

% for sub head 708 & 709 7% 

% for sub head 710 & 711 1% 

% for sub head 726 0% 

Avg. teacher salary : N.A. 

Est. total teacher cost : N.A. 

Est. % of MCE budget 

for primary teachers N.A. 



Box 14 



MOST RECENT YEAR'* *i991 



Total MCE budget 


378, 


719,680 








Total Dept. Budget 


3, 


613,780 


% for sub head 701^ 






% for sub head 702 




16% 


% for sub head 703 




2% 


% for sub head 704 & 


705 


4% 


% for sub head 706 & 


707 


0% 


% for sub head 708 & 


709 


1% 


% for sub head 710 & 


711 


4% 


% for sub head 726 




0% 


Avg. teacher salary 




N.A. 


Est. total teacher cost : 


N.A. 


Est. % of MCE budget 






for primary teachers 




N.A. 



Box 15 



Box 14 Notes: ^ 1989/90 Estimates of Expenditure from the Consolidated 

and Development Funds, Government of Botswana. 

^ 701 * Personal emoluments 

702 s Travelling & transport (internal) 

703 » Travelling & transport (external) 

704 « General expenses and supplies 

705 « Departmental Services 

706 « Maintenance & running expenses (equipment) 

707 s Maintenance & running expenses (other than equipment) 

708 s Institutional running expenses 

709 « Government hospitality 

710 « Training 

711 s Councils, conferences & exhibitions 
726 « Special expenditure 

^ Percentages less than 1% are listed as 0. 

Box 15 Notes: " Source: 1991/92 Estimates of Expenditure from the Consolidated 

and Development Funds, Government of Botswana. 



^ 701 




Personal emoluments 


702 




Travelling & transport (internal) 


703 




Travelling & transport (external) 


704 




General expenses and supplies 


705 




Departmental Services 


706 




Maintenance 6 running expenses (equipment) 


707 




Maintenance 6 running expenses (other than equipment) 


708 


m 


Institutional running expenses 


709 




Government hospitality 


710 




Training 


711 




Councils, conferences & exhibitions 


726 




Special expenditure 



^ Percentages less than 1% are listed as 0. 
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The Development of an Educational Information System: 

The Case of Nepal 



C. Howard Williams 



INTRODUCTION 

This paper is a reflective description of the development of an educational information 
system in Nq)al. The educational information system is comprised of a Statistical 
Information System (SIS), Action Research, Educational Indicators, and Evaluation 
Standards. In developing this system, an organization development approach has been used, 
in order to assess what educational information and systems would be of most value and use 
to the Nepal Ministry of Education, under present drcumstances. 

The Nepal Nfinistry of Education and Culture, for several pressing reasons, has given a 
priority to improving its capacity for collecting, analyzing, and using information. First, the 
history of education, and educational information in Nepal is a relatively recent one. A 
national system of schooling began in 1951 with 321 primary schools, and 11 secondary 
schools. Today over 13,000 primary schools cover most of the country. A National 
Education Plan (1971) introduced new curriculum, increased access through gradual 
elimination of tuition and textbook charges at the Primary level, and increased educational 
opportunities in rural areas. The secondary school system in recent years has been 
expanding somewhat faster than the primary, although even now it still serves less than 20% 
of the age cohort. Approximately ten percent of Nepal's youth complete grade 10 and, of 
those, some thirty*five percent pass the Secondary Leaving Certificate. 

This tremendous expansion of schooling has placed a heavy demand on the central ministry 
for developing and mamtaining adequate basic data on schools, teachers, and students, to 
make decisions such as allocation of teacher funding and textbook distribution. 
A second factor in making information a high priority is that the provision of schooling in a 
country such as Nepal is a very loosely coupled enterprise. The rugged terrain, extreme 
seasonal weather, and a communications and transportation infrastructure which is still in the 
process of being extended to large population areas, leave much of basic and primary 
schooling subject to community initiative and support. Implementation of a national 
curriculum, teacher attendance and teaching quality, and student attendance are, in most 
cases, better known to the community than to MOEC planners and administrators. 

The unavailability of this level of information makes policy and program development 
problematic, both in having an information basis for quality improvements and in following 
the effixts of policy and program decisions. The lack of information places an additional 
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constraint on the Ministry in its ability to effectively direct and coordinate donor assistance in 
education. If a donor agency proposes program assistance in education, based on studies 
conducted in Nepal by that agency, or on the basis of studies and experience in other 
countries, it is quite possible that the Ministry will not have available ''countervailing" infor- 
mation or data of its own with which to negotiate and sh2^ the donor proposal. 

A third factor drivmg the need for more comprehensive and higher quality data is the 
democratization of Nqnl's government. For approximately six months, beginning in the 
Spring of 1990, Nepal uperienced considerable political and social unrest. A new order was 
established late in the year with the adoption of a new constitution, based on democratic 
principles, a multi*party system, and a constitutional monarchy. In May 1991, the first party 
political elections were hdd since 1958. The Nq>ali Congress Party won the election with a 
simple majority. 

The first parliamentary session of the new democracy has recently concluded. A central and 
priority focus of the first session is on extending and improving support and services to 
Nqpal's rural poor. Nepal is a village-based society, with 91% of Uie people living in rural 
areas. Development priorities are to raise the standard of living of these people, with a 
commitment of 70% of the central budget for rural development. In the newly democratic 
system. His Majesty's Government has many new opportunities to address the challenges of 
Nepal development. In order to meet these opportunities, better data will be needed. 

The main source of educational information that the MOEC has relied on, to date, has been 
its own Statistical Information System (SIS), which is administered in the Manpower and 
Statistics Section of the Planning Division. 

In the Fourth Development Plan of Nepal, the Division of Planning was created to handle the 
data functions of the MOEC. Regular publications of Educational Statistics began in 1970- 
71. Over the past two decades there have been many changes in the reporting of Education 
Statistics, and improvements in the type of data reported, in format, and publication 
schedules. 

Under Phase I of the Improving the Efficiency of Educational Systems (lEES) Project 
(1985*89), an TFKS resident advisor was assigned to the MOEC from February 1986 to June 
1989. During that period a sustainable data collection and analysis capacity was developed 
within the MOEC's Planning Division. A focal point, now that data have become so critical, 
• is to what degree are the MOEC data valid and reliable, and whether the data address the 
critical, priority areas of educational efficiency and improvement. To expand such capacity 
and bring it more directly to bear on the MOEC's planning activities, AID/Nq>al **bought 
into** lEES Phase n (1990-94) for the services of amrther resident advisor for two years 
(1991-93), 

Phase n of lEES assistance in Nepal is focused almost exclusively on concentrated 
development of the MOEC's education information system, specifically on establishing an 
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''Education Management Information System** (EMIS) and conducting policy-oriented 
research. In this case of devdoping an information system, it may be useful to distinguish 
between a basic Statistical Information System (SIS), and an Education Management 
Information System (EMIS). 

The reference to **EMIS** in the agreement for TRRS Phase n assistance is probably an 
acknowledgement of the large currency **EMIS** has today in educational development 
efforts, periis^s as much as it may reflect what is actually expected from this assistance 
effort under the present circumstances. An EMIS would provide regular, timely information 
on project or program status and implementation for management monitoring, decisicms and 
actions (USAID, 1990). This type of system requires a fairly deq> level of trained personnel 
and technology which will be held online for the purposes of a particular project or program. 
The costs of dedicating the requi^te staff and technology to a selected effort, in the case of 
Nq>al, must be weighed against a larger cluster of information needs which presently face 
the Ministry, along with its csq^acity for generating, analyzing and using the information. 
For this reason, a more general "Education Information System" (EIS) is being developed, 
which incorporates the Statistical Information System (SIS), a set of educational efficiency 
indicators, an action research program, standards for die conduct for program evaluation, and 
an analytic csq)acity for ad hoc requests, which may be expected in a new democracy. * 

An information system must be technically sound and usable. Equally important, it must 
produce information which is valuable and useful. The dimensions of (a) technically sound 
and (b) valuable and useful represent an important distinction. 

The need for information systems which are technically sound is supported by experience in 
conducting educational research and developing EMIS's throughout the world. Chapman 
(1991) points out that donor agencies and ministries of education have recognized the need 
for better education data for improving quality and efficiency. USAID/Bureau of Science 
and Technology (1990) has produced a "Lessons Learned" review of experience in EMIS 
development in Third World Countries, based on work done by Moses and Thiesen. 

The dimension of value and usefulness draws more from nperience in design and 
implementation s^roaches and processes. In cases of technical assistance, the need for an 
interactive relationship, with the intent of reaching a valuable and useful product, in this case 
an informaticMi system, has been described by Adams and others (Comparative Education 
Review, 1990). Lind (1991) has raised this issue expressly in regard to designing and 
developing useful EMIS' in the Third World. 

Conceptual and practice-based guides for successfully engaging in a consultative, interactive 
relationship for agreed upon ends may be found in the organization development literature. 
Applied experience in using an organization interventicmist qyproach to educational planning 
has been described by Ciesswdl and Williams (1991). 
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It may be helpful, at this point, to make a brief reference to the Psacharopoulos*Adams 
dd)ate (1990) on what constitutes valid and useful information for educational planners. It 
seems that Psacharqxnilos' positi(Hi can be summarized as ''good research produces sensible 
results, which sensible pec^le ^licy makers and planners) should use/ And, it would 
seem, Adams' respmse is that "good research produces results which are sensible and 
value^le to certain persons under cotain circumstances/ 

Psacharopoulos' position, that substantive lesscms are compelling ones, is representative of 
Chin and Benne's (1976) characterizati(Mi of the rational-empirical strategy for change. 
Why, then, would we need to spend the additional time and energy on diagnosing and 
engaging in the organization itself if sufficient attention and skill has been given to 
developing a knowledge base on educaticmal effectiveness? Adams' call for interactive 
processes is representative of the normative re-educative model for change, but goes deeper 
into grounding the planning of change in the culture of the existing organization and 
participants. It is at this level that ''value and usefulness" of data, or a larger organizational 
intervention may be best informed. 

The interactive relationship among the lEES advisor(s) and the Ministry of Education and 
Culture, in this present case, has been guided by models and concepts borrowed ftom tiie 
organization development field* This approach to developing the education information 
system has produced a configuraticm of several types of ' information systems," e.g., 
research, evaluation, and statistics* Each of these components of the information system is 
recognizable by its own conceptual base, methodology, and techniques. Yet no single 
component is a wholly complete, stand-alone operation, aside from the basic SIS akeady 
established by the Ministry. Instead, the consi^tative, interactive relationship has produced a 
configuration of information functions and activities which s^pear to be the most valuable 
and useful, under the present circumstances of Nepal's Ministry of Education. 

In using the OD approach to developing the information system, three activity areas have 
been central: an on-going diagnosis of the system, team building, and action research. These 
three activities are interdqpendent, and have been implemented accordingly* 

A diagnostic stage has been a necessary prerequisite to engaging in a consultative relationship 
for development of the information system. This diagnosis has been carried out at two 
levels. An initial series of diagnoses was conducted under Phase I of tpfs (198S-89). 
During this poiod, OEES provided assistance through (a) a long-term advisor to the 
Manpower and Statistics Section (3 years), (b) an Education and Human Resources Sector 
Assessment, and (c) a research activity on the existing information system at the Ministry. 

The lEES Phase n diagnosis has been more of an "emergent" design, as it relies primarily 
on the long-term presence of the advison This approach is necessarily unstructured for three 
reasons: 
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1. The agreement with the MOEC is for (a) development of an EMIS and (b) the 
conduct of research. It is not an agreement for "organization development," as such. 
Consequently, there is no prior understanding between TEES and the MOEC for 
utilization of specific OD goals, references, or activities, beyond development of the 
Ministry's Information System and the production of research results. The OD 
concepts, instead, are used as guiding references for the approach in accomplishing 
the EMIS and research objectives. 

2. At present, there do not seem to be a ready set of experience^based OD models for 
designing and implementing educational system interventions in the Third World. 
This would be in particular reference to effective engagement with an existent 
educational organization so that the intervention constitutes organizational 
improvement (in effectiveness or efficiency) beyond the project deliverables. The 
problem of engagement has becm clearly identified by what are now common 
references to "transfer,* "capacity," and "sustainability." These references 
acknowledge the need for organizational ownership over the program and/or the 
product. The OD s^proach, in a sense, "front-loads" the ownership to the very 
beginning phases of diagnosis and design, so that the organization will not take on 
what it doesn't value or can't use. The development or adaptation of OD models for 
educational intervention would seem to be a useful contribution to educational 
development. 

3. The long-term presence of an advisor allows for interactive participation throughout 
the process of diagnosis and design of the intervention. The diagnosis can "afford" to 
be open, following where the opportunities lead for determining organization 
functions, roles, culture and values, i.e., those things which make or break an 
intervention. The consistent thread of continuity through this open, collaborative 
process has been to develop the EMIS and conduct the research. To the extent that 
we have focused on value and utility of those products, we have entered into an OD 
approach to determine the role, shape, and nature of mformation so that it will be 
valuable and useful. (And, if it is valuable and useful, it is more likely to be 
sustainable.) 

This ongoing diagnosis has attempted to make the conditions of the Ministry more known 
and discussable in designing, planning, and implementing the information system. 
CcHisultation among the advisor and system participants has helped to assess the prevailing 
climate and culture, and how the system can best be supported by, and serve the various 
individuals and groups in education. 

Orggnizatioiial diagnosis: conditioiis for saccess in using an OD approadi. In 

undertaking an OD approach for develq[>ment of the Ministry's information system, it was 
important to assess whether the conditions in the Ministry were conducive or constraining to 
using an OD approach. French and Bdl (1984) outline twelve conditions which they see as 
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consistentty important to "successful evolution of organizational development efforts'* (p. 
215)- 

The conditions which are cited by French and Bell are considered important contributors for 
optimal success for an OD effort. As previously stated, the use of OD in this case of 
developing the I^Gnistry's information system has been primarily as a guiding set of 
references for effective intervention. It is felt, however, that reflection on these conditions 
for a successful, full-blown OD effort is still useful m gauging whether the OD approach was 
likdy to be acc^ted as an appropriate means of doing business, and the likelihood of success 
in using OD constructs, such as action research and team building. 

Following each condition statement, below, is a summary assessment of that condition in the 
Ministry of Education, as perceived by the lEES resident advisor. As the resident advisor 
has taken an interactive, consultative approach to diagnosis, design, and development, these 
conclusions are based on numerous descriptive and reflective discussions with Ministry 
officials. This paper, itself, will be reviewed by members of the EMIS Committees and 
research team as part of the ongoing, consultative relationship for development of the 
information system. 

1. Percepdons of orsanizational problems by key people and perceptions of the 
relevance of the bdiavioral sdences in solving these problems. Key Ministry 
officials (Secretary, Additional Secretary, Joint Secretaries), other HMG officials, and 
prc^esson from the national university have commented, without dissension, that the 
SIS needs strength^iing, particularly in validating school level record keeping and in 
refining data definitions. 

Following N^al's democratization, there is now the additional opportunity to use 
information for policy and program decisions below the level of the Palace, a 
condition which is new to the Ministry. Pressure &om the Parliament has been joined 
with this new opportunity, as parliamentarians have already begun asking what tiie 
specific deliverables are, from the Ministry to their constituent districts. These 
factors have focused the Ministry's attention to developing and maintaining a more 
comprehensive and higher quality information system. 

The means for developing a more responsive information system is acknowledged by 
kty participants to be an organizatiiHial issue, over and above the technical 
requirements for accomplishing it. 

2. The introduction into the system of a bduvioral scientist-consultant. lEES is 
somewhat unique, from the percqptions of hfinistry officials, in that its main input is 
the placement of a resident advisor for a two-three year assignment with the Ministry 
of Education. The Ministry is more accustomed to larger projects which provide 
more capital investment in the system. The accqptance of the lEES packayge 
constitutes an acceptance of the internal consultant. The former and current lEES 
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resident advisors have had training and experience in administration, planning, and 
evaluation. 

Initial top-tevel mvolveiiient, or at least support from a higher echelon with 
subsequent top-managcanent involvement. The Permanent Secretary of the Ministry 
has been an active supporter of educational information systems development, and 
TRRS assistance to that effort. The Secretary served previously as Joint Secretary for 
Planning, and Additional Secretary for the Basic Needs Program, an overall program 
initiative of the Ministry. The Secretary chairs the HMIS Steering Committee, and 
has been a leader in the development of the Ministry's SIS and research agenda. The 
Secretary also recently chaired a two-day inter-Ministerial workshop on the 
Ministry's information system. The seminar was co-chaired by the Vice-Chairman of 
the National Educaticm Commission, which is tasked with development of the next 
National Education Plan for submission to the upcoming parliamentary session. 

The Additional Secretary, and Joint Secretaries for Primary Education and Planning 
have been active collaborators in the identification of priority issues and subsequent 
design of information systems activities and development, lliis input has been stated 
by them to be guided by their need to use the information, and the need for it to be of 
high quality. 

Participation of intact work teams, including the formal leader. The Manpower 
and Statistics Section of the Planning Division has been the primary group responsible 
for the SIS functions of the Ministry. This group now comprises Uie main working 
team of the EMIS Technical Committee, which handles the day-to-day development 
functions of the EMIS. The information system design and activities are the result of 
section staff (EMIS technical Committee) analyses, deliberations, and planning, in 
consultation with the Joint Secretary for Planning and the Secretary. 

The Under-Secretary of the Manpower and Statistics Section (section head) serves as 
member secretary for the EMIS Steering Comnuttee. The Joint Secretary for 
Planning has served as Chair of the EMIS Technical Committee. These assigned 
roles are consistent with the normal functions of these positions; the more recently 
assigned EMIS roles provide greater focus and explicit responsibility in regard to the 
information system. 

The dual roles of the Manpower and Statistics Section staff (MOEC proper, and 
EMIS) are acknowledged and supported by senior members of the Ministry. The 
Section staff are called upon regularly for briefing of senior Ministry officials and the 
National Education Commission, in regard to educational statistics and the 
information system activities. 

The operationalizing of the action research model. Action research constitutes one 
of the three main activity areas in OD. Action research, as described by French and 
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Bell (1984) replicates the steps of the "scientific inquiry," with a particular focus on 
the processes of data specification and collection, feedback of the data to the client 
organizati(Hi, and subsequent acti<xi planning. Schein (1980) adds that any assessment 
of the oi^anization "made to evaluate the effects of a prior intervention itself becomes 
automatically the next intervention" (p. 242). 

French and Bell (1984) emphasize that action planning is both an approach and a 
process. French and Bell cite Corey's (1953) reference to action research in 
education as "the process by which practitioners attempt to study their problems 
scientifically in order to guide, correct, and evaluate their decisions and actions (p. 
108)." 

Action research outcomes provide solutions to immediate problems, which form the 
basis for organizational action. This is an s^ropriate characterization of the research 
component of the Ministry's education information system. The most recent action 
research design in this case is, more specifically, "participant action research," which 
Chein, Cook, and Harding (1948) describe as involvement of the people who are to 
undertake the action throughout the entire research. This participation increases the 
likdihood that the recommendations are "feasible and workable" and that the actions 
are more likely to be carried out. (French and Bell, p. 116). 

The action research agenda associated with the development of the information system 
has occurred in three phases. The first action research activity was conducted under 
Phase I of the lEES Project. This research was focused on the Ministry's information 
system, documenting its history, present status and characteristics, and quality. The 
results from this research on the EMIS included a finding that the school level data 
were reported ftom the schools and entered at the central Ministry at a high rate of 
accuracy (95%) (New Era, 1989). The results of this action research were reported 
at a national seminar. 

A second action research phase consisted of three micro studies on the SIS focusing 
on (a) current issues in today's information system (see Attachment A), (b) rates of 
primary school dropouts and repeaters, and (c) a needs assessment survey of SIS data 
users in the government and the University. The results of these three studies were 
rq>orted in a two-day workshop for discussion by participants from government 
agencies and the University. 

As part of this workshop, a third action research proposal was introduced for 
discussion, which was the result of consultation among Ministry officials and lEES 
representatives. This research is focused on investigating the (a) family, school, and 
community factors contributing to grade repetition and student attrition in Grade One 
and (b) what actions parents, school personnel, and community leaders believe can be 
taken at the community level and by the Ministry to improve student flow in Grade 
One. 
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The design of the study is grounded in two premises. First, grade repetition and 
dropout at the end of Grade One rqnresents one of the most serious constraints to 
internal efRciency now fiadng the education sector in Nepal. Second, much is ahready 
known about the reasons for first grade repetition and dropout. However, much of 
the solution for improving student flow will rest on int^entions undertaten at the 
community level. More information is needed on community-school dynamics if 
effective assistance is to be developed for communities to raise the ef^dency of 
primary education. 

The anticipated benefits of this research activity are that the Ministry will have a 
better understanding of the community level dynamics that might contribute to 
improving student flow. Second, the design offers a model for action research that 
the MOEC can use to investigate the impact of interventions aimed at solving other 
issues in educational quality and efficiency. The research activity will begin by 
March, 1992 and is expected to be concluded in February, 1993, 

Early successes^ with expansion of the effort stemming from these successes. The 
Manpower and Statistics Section staff had met with early success in improving the 
SIS, with more timely publications of annual statistics and a positive assessment of the 
reporting system from the initial action research activity. Although the validity of the 
school level data is still in question, that is perceived to be a ^'next step"" in 
strengthening the system rather than being a fsulure from past efforts. 

A high profile success was achieved during Phase I of lEES in the conduct and 
publication of the Education and Human Resources Sector Assessment (1988)- The 
Sector Assessment Team included MOEC officials, faculty from Tribhuvan (national) 
University, and lEES Project technical assistants. The Sector Assessment document, 
now four years old, still serves as a basic information reference for the Ministry. An 
accomplishment in early work of the EMIS technical group came through conducting 
a cost analys^ of providing free secondary schooling. Team review and cross- 
examination of cost categories and assumptions demonstrated that the liability to the 
government Would be at least 500% greater than earlier estimates. As a consequence, 
the ruling pi^ chose incremental implementation of its free secondary school policy, 
starting with free grade six. This action provided a powerful impetus to tiie EMIS 
technical working group, in tiiat its data and analysis could be seen to have a policy 
impact, even' at tiie Parliamratary level, 
t , 

The EMIS Technical Committee and Steering Committees have also met witii success 
in developiag inter-group working relations, especially witii ttie University. The 1991 
workshop on the Ministry*s information system included many participants within the 
Ministry, pom otiier Ministries and the University. The workshop produced 
thoughtful, workable advice on tiie SIS, and the research. It also demonstrated tiiat 
the MinistiyHSould open its internal information program to persons outside of the 
Ministry and obtain valuable and useful feedback and guidance. 
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7. An opeiit educational philosoj^y about the theory and the technology of OD. 
The TEES resident advisor has been opm throughout in discussing objectives, 
activities, and ratimales. This is particularly the case as all programmatic action is 
arrived at collaboratively and is taken as a Ministry initiative, generally sponsored 
through its EMIS Steering or Technical Committee. 

Discussion about OD, however, has been limited to focusing on the strengths of the 
work teams, the inter-group work teams, and the action research, which are key 
components in an OD program. Since the OD approach used in this case is 
incremental and evolutionary, attention is given to open, educational discussions about 
the benefits and disadvantages for each initiative or proposed action. If we proceed, 
modify, or abandon an idea or proposal, it is a considered Ministry decision. 

8. Acknowledgeoient of the congniency between OD and many previous eflTective 
management practices. There are several congruencies between the OD-type 
activities which are being used in developing the information system and in practices 
already used by the Ministry. The use of work teams is a long tradition in the form 
of "steering committees" for project assistance and activities. The focus in this case 
is somewhat less on referent advice and more on active participation in the 
development agenda. 

The action research model lies somewhere in-between traditional practices of 
Ministry-sponsored evaluation and University-initiated basic or applied research. 
Effective practices of each mode, the organizational improvement function of 
evaluation and the theory and model building of university research, are incorporated 
into the action research model. 

Thus &r, the team building, intergroup team building, and action research have been 
supported by the Ministry Secretaries as consistent with their ideas for improving and 
managing the educational system. 

9. Involvanent of personnel and industrial relations/human resources management 
people and congniency with personnel policy and practice. There are presently no 
official positions which are q)ecifically charged with responsibility for internal 
personnel/human resource deveiopmrat. 

In order to maintain consistency betwem the OD-type activities and present personnel 
policies, there have been no assignments or realignments of roles or responsibilities 
for participants which would be inconsistent with current assignments. Instead, the 
focus in negotiating work roles and assignments in developing the information system 
has been to enhance current job roles and performance, in many cases providing an 
avenue to punue what is abeady charged to the individual or section, but may not 
have been taken up in the absence of a clearly defined focus or product. In this way, 
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it is expected that unrealistic or dissonant expectations are not being set, and that the 
developniratal activities are building upon existing authority and mandates. 

10. Develoiwieiit of internal OD resources and facilitative skills. This has been done 
formally on one occasion* Three Manpower and Statistics Section officers and the 
lEES resident advisor participated in a three-day organization development workshop 
in Kathmandu, conducted by a professional OD consulting lirm, The M^tor Group, 
based in Seattle and Hong Kong. The four participants gained insights and skills in 
agenda setting, establishing effective work groups, and in analyzing organizational 
problems and prospects for action. A secondary braefit of workshop participation 
was a further soli^cation of the EMIS technical working team, as the seminar 
allowed for group work with specific reference to actual work situations. A follow- 
up workshop is scheduled to take place four months later to reflect on and analyze 
changes in the work situation and personal effectiveness, and to reinforce the 
knowledge and skills presented in the original workshop. 

Facilitative sidlls are bemg mformally developed and reinforced by these and other 
members of the information system, particularly the action research team, which has 
members from the EMIS Steering and Technical Committees, the ^^niversity, and a 
private research firm. 

At present, however, there are no plans for additional formal training in OD and 
facilitative skills. 

11. Effective management of the OD process and stabilization of changes. Since the 
relationship between the resident advisor and the Ministry officials for information 
and planning are interactive and collaborative, positive and negative implications for 
initiatives and proposals are discussed at length before action is taken. 

Changes are stabilized, in part, because the working members of the EMIS Technical 
Committee are personally conscientious in producing results which are useful for the 
Ministry, and prefer efficacy to change for its own sake. Change, in some cases, has 
not resulted in productive outcomes, causing frustration and disappointment. These 
cases have been discussed to refiect on what could have been done differently, or if 
the activity should have been undertaken at all. Refiective discussions have been 
usefril in projecting implications for proposed activities, both in considering 
organizational costs and benefits, and in designing a procedural strategy. 

12. Monitoring the process and the measuring of results. In the present case, this is a 
tough one. There is no pre-existing agreement on OD changes or effects, beyond 
establishment of an information system and the conduct of research. Monitoring and 
measuring OD progress, as a cumulative result of the OD approach will tend to be 
refiective and anecdotal. Surfacing the underlying concepts which have guided the 
OD sqyproach in this work, through this paper, and discussing them with the 
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information system participants may serve as a first step in determining whether any 
OD changes may be specified in measurable terms, or whether that would be valuable 
or useful for the Ministry. 

In the above discussion on diagnosis and development of the educational information 
system^ the Statistical Information System (SIS) and Action Research components 
have been introduced and described. Two additional components, Educational 
Indicators and Evaluation Standards, will be described below. 

National educadonal indicators. A proposal and draft set of national educational 
indicators was developed for review by the participants in the 1991 national two-day 
workshop on the Ministry's information system (see Attachment B). 

The indicator set was proposed for use in organizing data for assessing the status and 
performance of education in Nepal, monitoring important developments, and showing 
trends in key dimensions of educational improvements. When the indicator set is 
established, it may also be used as a reference for the MOEC in prioritizing research 
and routine data collection by projects and programs, to fill essential gaps, avoid 
duplication of effort, and to meet the highest priority data needs of the Ministry: 

To determine which data can serve as indicators, a series of considerations will be 
addressed: 

a. Are the data useful in telling us something about the 
educational system? 

b. Are the data sufficiently representative? 

c. Are the data available and collectible at reasonable cost and effort? 
Similarly, we will also address several programmatic considerations: 

a. Who should collect the data, and at what level? 

b. Who should analyze the data, at what level? 

c. How oftoi do we need to collect the data? Will sample data serve adequately, or 
is census data necessary? 

d. Should the MOEC serve as primary collectors of the data or can we use 
secondary sources? 
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The ''indicators ^loach'' was based on review, compilation and analysis of indicators 
research conducted at Florida State University (Eastern, Holmes, Williams, and duPlessis, 
1990) and refined in consultation with the Mhustry*s EMIS Technical Committee. 

The Secretary and other officials expressed their view that the indicators model was 
particularly helpful as a ccmceptual reference, in seeing where the strengths and weaknesses 
of the data system lie. The model was also helpful in discussions for further information 
system development and activities. For example, ccmsultation with Ministry officials on the 
topic of M0EC7IEES action research focused initially on school processes, which tends to be 
the most difficult domain to observe and collect meaningful data. It seemed, then, that this 
domain would be a good candidate area for action research, collecting valuable and useful 
data on school processes most of interest to the Ministry, e.g., teaching quality. 

The Secretary and other officials acknowledged that the area of school processes is a weak 
area in terms of data and knowledge on schooling, and that targeted research, using careful 
sampling, was needed in this area. The focus of discussions on research, however, shifted 
more to the "action" aspect of the research, that is, the topic for which the Ministry felt the 
most urgency to address as a naticmal issue in schooling. The Secretary and others expressed 
their interest in learning more about the phenomenon of student dropouts and repetition, and 
their concem that these students represented a tremendous cost without related rates of 
success and attainment. Teaching quality was agreed to be an important area for research 
and improvement, but was felt to be somewhat secondary (though not unrelated) to effective 
student partidpation and progression. 

Consequently, it was agreed that the research would focus on family, school, community 
factors which contribute to, or inhibit, student enrollment, retention, and flow. In addition, 
the "action research" bias placed an additional emphasis on generating community-based 
recommendations, or "solutions," to the phenomenon of dropouts and repetition. These 
solutions, or case recommendations, arc expected to focus and guide what tiie Ministry, or 
His Majesty's govemment, can do to support communities in their support of schooling. 

Eyaluation standards. The evaluation capacity of the Planning Division will be assessed 
and enhanced as a part of tiie EMIS worlq)lan. The organizational capability of ti\e central 
Ministry to conduct extensive evaluations of all programs and projects is not likely to be in 
place in die foreseeable future, when weighed against other competing program needs. A 
preliminary "consultative analysis" of the evaluation functions of die MOEC suggests that the 
most fruitM approach may be for the central MOEC to develop program evaluation 
standards, along die lines of standards developed by die joint American Educational Research 
Association/American Evaluation Association Committee and adapted by the State of Florida 
(1986) and the Govemment of Botswana (1988). 

Widi reference to an established set of evaluation standards, the actual conduct of evaluation 
may then be delineated by function and purpose: formative evaluation may be conducted by 
donor-assisted initiatives for program and project management purposes, and summative 
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evaluations may be conducted by the central Ministry with technical and financial assistance 
from the donor-assisted projects. 



CONCLUSION 

The educational information system which is being developed for Nq)al is idiosyncratically 
comprised of a Statistical Information System (SIS), Action Research, Educational Indicators, 
and Evaluation Standards. In developing this system, an organization development approach 
has been used, in order to assess what educational information and systems would be of most 
value and use to the Nepal Ministry of Education, under present circumstances. This 
reflective description of the ^roach used, and activities undertaken will itself be used in 
this interactive, consultative process for developing the system. 
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APPENDIX A 



ISSUES IN TODAY'S EMIS 

L RELEVANCY 

•Is the data which we are collecting and reporting meeting the needs of the data users? 

•Are there different frameworks and formats we can use for reporting? 

•Are there any special studies or data collection activities we can do? 

•Are we collecting the necessary data, and only the necessary data? 

•Is the frequency of our data collection appropriate or not for each type of data? Are 
we using census data when sample data wiU be as useful? 

2. ACCURACY 

•The ENQS study by New Era indicated that we are accurate within 5 % in transmitting 
data from the school records to the MOEC Manpower & Statistics Section for analysis 
and reporting. 

•Our analyses (such as gross and net enrollments) give us some concerns about the 
accuracy of the data. Are the schools recording and keeping accurate data? 

3. TIMELINESS (UP-TO-DATE) 

•If the data are seriously delayed, it detracts from their useMness, Are there problems 
in the EMIS data process which slow down the data collection and reporting, which we 
can improve? 



84 



105 



4. DECENTRALIZATION OF THE EMIS 



•The system presently provides copies of the district summary data to the DEIs and 
REIs. What can we do to improve data activities and flow at all levels? 
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5. PRIVATE SCHOOL REPORTING 

•According to district reports, private schools are not very responsive in reporting to the 
DEIs, and we have very little leverage over these schools. What can we do to improve 
private school rqwrting? 
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APPENDIX B 



EDUCATIONAL INDICATORS 

The M&S Section of the MOEC's planning division has already established data collection and 
reporting on several key indicators: 

I. INPUT INDICATORS 

1* School endowment 
a* number of schools 

2. Student characteristics 

a* total enrollments 

3. Teacher characteristics 

a. teachers* level of training 

n. PROCESS INDICATORS 

4. Curriculum quality 

a* subjects taught 

5. Student participation 

a* drop-out rate 
b* repeater rate 

in. OUTPUT INDICATORS 

6. Student attainment and achievement 

a. number of students reaching the last grade of cycle 

b. student grade-to-grade progression rates 

c- level examination results (grades 5,7, and 10 (SLC) results) 
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IV. EQUITY 



a. students and teachers, by region, by sex. 



Suggested MOEC Education Indicators (by domain) 



I. CONTEXT INDICATORS 
L Socio-economic and demographic background 

a« total/regional population 

b. life expectancy at birth 

c. purchasing power per household 

d« newspaper/radio circulation per 1000 population 
e« adult literacy rates 

f . number of hours per da. spent on labor for school 
age children 

g. family preference for educating boys vs. girls 

h. career expectations for boys and girls 

i. evidence of community poUtical participation (e.g*, 
percent) adults casting votes in elections; percent 
adults attending community meetings 

j. number of active community civic organizations 



n. INPUT INDICATORS 

2. School endoMment 

a. number of schools 

b. class to classroom ratio 

c. whether fumiture is provided 

d. the availability of instructional materials 

3. Local/commumiy support 

a. building of school facilities by the community 

b. maintenance of school facilities by the community 

c. whether a functional parent/teachers committee or 
association exists 
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d« the nature and frequency of community-school 
contacts 

4. Central/regional/district support 

a* the number of textbooks produced 

b. the number of school inq)ection visits by MOEC 
officers 

c. the overall and HMG/MOEC non-personnel expenditures 
on schooling 

5. Student characteristia 

a. total enrollments 

b. the percentage of female students 

c. calories consumed per day per person 

d. health status of school age children 

6. Teacher characteristics 

a. the number of teachers 

b« the percentage of female teachers 

c. qualification: training (by level) 

d« average salaries 

e« attrition rates 



m. PROCESS INDICATORS 

7. School numagement 

a* number of school days per year 
b« hours of instruction per day 
c« teacher-studwt ratio 

d. school expenditures on £Eu:ilities and maintenance 

INDICATORS OF HEADMASTER CHARACTERISTICS MAY INCLUDE: 

e. number of headmasten completing administrative 
training 

f. level of teacher training 

g. years experirace 

h« amount of time spent in classroom instruction 

88 



ERLC 



109 



i* amount of time spent on supporting other teachers' 
instruction 

j\ amount of time spent in introducing or supporting 
instructional innovations 

8. Curriculum quality 

a. subjects taught 

h. professional judgment of the ^ropriateness of the 
curriculum 

c. parents' judgment of the ^ropriateness of the 
curriculum 

d» the degree to which the national curriculum is 
followed 

9. Teaching quality 

a* the amount of teachers' time spent on instruction 
the use of instructional materials for instruction 
c* the frequency of testing and feedback to students 

10. Student participation 

z. average daily student attendance 

b. student time on task 

c. dropH)ut rate 

d. repeater rate 

IV. OUTPUT INDICATORS 
1 L Student attainment and achievement 

a. percentage of students reaching the last grade of 
cycle 

b. student grade-to-grade progression rates 

c. results of level exams 

12. Student attitudes and aspirations 

a. the number of students seeking admission into the 
next level of schooling (grade 5 into secondary 
schools) 

b. the general level of occupational choice for school 
leavers and school completers 
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V. OUTCOME INDICATORS 



13. Lcaer academic outcomes 

a* percent of graduates of the primary cycle entering 
the following cycle 

the number of students mtering/graduation from 
scientific and technical fields 



a* labor force status of cycle graduates 
b. average earnings of cycle graduates 
c* average agricultural productivity of cycle graduates 

IS. Social and political outcomes 

a. numbtf of primary school graduates voting in 
elections 

b. number of primary school graduates participating ir 
civic organizations 

c. social mobility of primary school graduates, e*g., 
positive changes in land ownership, income-debt 
ratios, capital investments, etc. 

16* Feedback to context 

a. recent primary school leavers' aspirations for 
family— size, economic, and educational aspirations 

b* recent primary school leavers' participation in 
community affairs 

c. recent primary school leavers' support for 
community's schools 
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Educational System Performance Indicators as they 
Apply to Curriculum Development and 
Implementation Policy and Planning 



Kent L. Noel 

INTRODUCTION 

Many of the thoughts discussed in this pap^ are based upon my experience with the Junior 
Secondary Education Improvement Project (JSEIP)) a large-scale education project jointly funded 
by the Government of Botswana and the United States Agency for International Development 
(US AID) ftom 1985 to 1991. V/hile the tasks of my job as an instructional systems design 
advisor with the project spanned all levels of Botswana's educational system, most of my work 
was concentrated within the Department of Curriculum Development and Evaluation (CD&E). 
Within CD&E, I worked with officers in planning, designing, developing, evaluating, and 
implementing curriculum and in establishing linkages among the units within CD&E and between 
that department and other Ministry of Education departments. While, a number of my 
suggestions may be specific to the operational context of Botswana's educational system; it is 
hoped that they may be of use in other educational contexts and systems as well. 

I recently participated in a number of work sessions with other USAID education advisors and 
planners in Botswana to draft some educational targets and indicators for a strategic educational 
objective of the USAID mission to be achieved during the next several years. The objective was 
related to a long term educational project being planned with the Government of Botswana and 
was to increase Batswana student skills and competence in order for them to better participate 
in society. Two main targets which were seen as contributing to the strategic objective were 
derived. One target was to enhance the quality of curriculum and instruction. Another was to 
establish systems to provide feedback on student learning achievement to the major stakeholders 
of the system. Related subtargets were to institutionalize appropriate processes and mechanisms 
required to support and sustain quality curriculum, instruction, and evaluation. 

As one might expect, some of the indicators identified to measure the achievement of the 
strategic objective, targets, and subtargets primarily dealt with outputs like the number of 
classrooms using new curricular materials, numb^ of teachers trained to teach the new 
curriculum, number of subject areas using criterion-referenced examinations, and number of 
classrooms in which continuous assessment was being used. 

Less typical and more important to this discussion was an attempt to identify indicators to 
measure quality in the curriculum development process itself. These included checklists of 
major functions and skills required by local officers working in the curriculum development sub- 
system. An assumption was that, by taking yearly "snap shots" of personnel performance of 
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items on the checklist, one could monitor officers' performances and, hopefully over the time 
of the project, see an increase in their abilities to operate, sustain, and improve the sub-system 
in which they were operating* 

It would seem that an assessment of the curriculum impact of large-scale educational projects 
should always include measures of how curriculum development and implementation sub-systems 
function, not only in terms of the products that are produced but also in terms of the processes 
that are employed. This would seem especially appropriate when the institutionalization of 
systematic curriculum development processes are among the project targets. However, 
assessment of the impact of project curriculum components is not routinely built into the original 
plans of large-scale educational projects. The plans for educational projects in general tend to 
focus on products and ouQ>uts of curriculum development sub-systems and student achievement 
indicators as a measure of project impact on curriculum sub-systems. They rarely pay much 
attention to measures of the effectiveness of curriculum development processes themselves. 
Also, current educational performance research provides little guidance as to how to gauge 
quality and efficiency in curriculum development sub-systems and the processes they use. Nor 
does research provide insight into what combination of sub-system characteristics might have the 
greatest influence on ensuring that useful curriculum development processes are introduced, 
used, and maintained beyond the life of a project. 

EVALUATION OF CURRICULUM DEVELOPMENT PROCESSES 

Educational management information systems (EMIS) have focused on indicators which help 
inform policy decisions at a macro level such as those made within a Ministry of Education's 
Planning Unit. This has included system-wide information such as numbers of teachers, schools, 
pupils, and curriculum materials that are in place and, more recently, school-level data such as 
indicators of student, teacher, and headmaster performance and attitudes. Educational 
performance research that addresses the information needs of mid-management levels and those 
charged with the processes of developing and implementing policy at sub-system levels seems 
to have been overlooked. At least in the case of Botswana, there has been very little information 
about the quality of curriculum or the processes required to ensure the development of quality 
curriculum (Burchfield, 1991). For example, within the context of a production-function 
framework for studying the performance of educational systems (Easton, et al. , 1991), indicators 
of quality and efficiency of the curriculum development sub-system have mostly dealt with the 
following categories: 

•inputs, such as numbers of officen which staff a curriculum development center and 
their level of efficiency; 

•products, such as the quantity of curriculum materials being produced and the number 
of teachers prepared to use the materials and to effectively implement the curriculum; 

•and outcomes, such as achievement levels of students who have used the curriculum 
materials. 
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Even when a ^'systems*' paradigm is used to evaluate an educational system, we have ignored 
process indicators of quality and efficiency used in developing the curriculum. Those processes 
include designing, developing, and evaluating curriculum materials and delivering and 
implementing those materials. Associated with each of the systematic curriculum development 
processes are general operaticmal processes and functions that are also important to the 
effectiveness of curriculum and development organizations such as management and supervision, 
communication and consultati(Hi, power sharing and lines of authority for decision-making, 
production,and delivoy. 

It is important that information regardmg curriculum development processes and organizational 
operations be collected, analyzed, and made available to curriculum policy decision makers for 
several reasons. 

First, in a centralized educational system such as that found in Botswana, the curriculum 
development component is a primary translator of national educational policy into concrete 
curriculum products. Research would focus much needed attention on the processes of the 
translation and might lead to a realistic assessment of the time and effort spent in the planning, 
tiaining, and organizational dspcets of curriculum development. With that information, 
curriculum policy makers could make better informed decisions about what curriculum initiatives 
could feasibly be introduced into the educational system and when they could be intioduced as 
well as the capacity of the current curriculum organization's capacity to handle those initiatives. 

Second, regardless of the level at which the technology is implemented (e.g., the school level 
or the central ministry level), it is likely that some change must occur within the curriculum 
development level and/or in the technology itsdf before it can be accommodated, 
institutionalized and sustained within the curriculum development sub-system. Research can help 
identify ways in which the curriculum processes or new educational technologies must be 
adapted in order to become functional. 

Third, like teachers and students, curriculum developers have characteristics, abilities and 
attitudes that affect their performances and they require incentives to stay on the job and to 
perform their jobs well. Research could help identify the qualities of a curriculum development 
sub-system which positively or negatively arffect officer job performance and, indirectly, affect 
the quality of curriculum products. 

Recognizing that "the provision of well-designed curricula and materials can be one of the most 
powerful interventions to improve ^«\ts' achievement" (Chapman, 1990, p. 5) and 
recognizing the importance of the curriculum developer's role within the educational system, the 
following section suggests types of information that would be helpful in monitoring and 
improving both curriculum policy decisions and the quality of curriculum development products. 
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WHO WANTS THE INFORMATION 



In setting up an EMIS, one must have in mind these questions: who wants information, what 
kind of infonnation do they need, will the information be used, and when? The ''who'' question 
is briefly discussed below because it can be addressed in general terms. The other questions will 
not be addressed here because their answers depend on the specific educational contexts in which 
they are being asked. 

The types of potential users of information on the functions of a curriculum development sub- 
system could include the following: 

1. Educational policy makers at the educational system level. Information on how 
educational sub-systems are functioning might have less relevance to "what" curriculum 
initiatives axe desirable than relevance to "when" those initiatives will be implemented. Useful 
information for policy makers would be the present capabilities and capacities of the sub-system 
to adequately handle current work loads and the projected capabilities and capacities required 
to handle new curriculum initiatives. 

2. Educational planners at the educational systan level. When a new curriculum policy is 
to be implemented, educational planners must be able to project what that means in terms of 
costs to the system. The most accurate way for them to do so is to look at cost information 
about similar initiatives that have been implemented previously. 

3. Curriculum decision makers and planners within the curriculum development sub- 
system. Executive officers and curriculum development ofRcers within the sub-system, guided 
by system-level policy, must also make curriculum decisions and plans to implement that policy. 
Therefore, they especially require information about their capacity to implement policy so they 
can budget their resources accordingly. 

4. Project planners with donor agencies. Project planners and developers within donor 
agencies must have a realistic understanding of the capacities of educational sub-systems. 
Making decisions based on this infonnation might ensure that the initiatives and agreements 
made with client govemmrats are realistic. It might also ensure that requisite resources are 
provided by the donor agency or the client government to guarantee a reasonable chance that the 
project will be fully implemented and that initiatives will be maintained after the life of the 
project. 

5. Project developers In the International conununity. To understand why an educational 
initiative in one country succeeded or failed and to predict whether similar initiatives might be 
successful in other countries and contexts, one should have an understanding of the conditions 
under which the initiative was devel<^)ed and implemented. Future projects targeted toward 
similar goals could be designed to ensure that conditions for success would be addressed. 
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COMPONENTS OF AN EMIS MODEL FOR CURRICULUM DEVELOPMENT SUB- 
SYSTEM 



Chapman states, "An EMIS consists of components: (1) a specification of data needs; (2) 
selection of ^ropriate indicators; (3) a procedure for collecting, coding, and sorting the data; 
(4) a system for analysis (manual or computer-based); (5) personnel trained to conduct the 
analysis; and (6) personnel who can interpret the information after it has been analyzed and 
relate the results back to the questions and policy issues under discussion" (1990, p. 227). 
Because of the preliminary nature of this discussion, this section and the next are limited to the 
first and second components. 

As a framework for listing the various categories of indicators, I have adapted the EMIS model 
used in Botswana which incorporates the components listed below. Because Botswana's EMIS 
model was designed to look at system-level components and the current discussion is looking at 
a specific sub-system, curriculum development, the definition of each of the components have 
necessarily been changed to reflect the environment and functions within which curriculum 
development operates. 

Context. Context information is drawn from needs assessments and situational analyses that 
describe how the curriculum development sub-system relates to other factors in its operating 
environment. It identifies the current status of the sub-system in terms of its strengths, 
weaknesses, and constraints. 

Inputs. Inputs are the resources available for curriculum design, development, production, and 
implementation. They also include human resources and their characteristics including those of 
curriculum development leaders, curriculum development officers, and officers in other 
components of the educational system (e.g., in service officers and school inspectors). 
Additionally, they include characteristics that are included in system-level EMIS including those 
of students, schools, teachers, headmasters, and instructional materials. 

Process. Because curriculum development is, by definition, a process, this component should 
be regarded as the heart of a curriculum development sub-system. It includes the processes and 
functions required to transform system-level policy into curriculum reform, including existing 
and new educational technologies that are used to implement that reform through the design, 
developmoit, and evaluaticm of curriculum materials and the implementation of those materials 
in the schools. 

Outputs. Outputs are the direct and immediate effects or products of the curriculum 
development sub-system. These would include curriculum nuiterials (e.g., student books and 
teacher's guides), methodologies required to teach the curriculum (e.g., child-centered 
approaches), pie-service and in-service teacher and officer training to implement the curriculum, 
and supplementary material and equipment requirements. 
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Outcraies* Educational outcomes are provided through the interaction of curriculum ou^uts 
with the school and social environment. Because a curriculum development sub-system is at the 
center of the educational system, its outcomes can be regarded as the ou^uts of the system as 
a whole* They include student achievement of learning objectives and associated attitudinal and 
behavioral changes. They may also include teacher skills, attitudes^ and behaviors. 

SUGGESTED INDICATORS OF CURRICULUM DEVELOPMENT EFFICIENCY AND 
EFFECTIVENESS 

The following list of indicators of curriculum efficiency and the types of information that would 
be useful to curriculum development and implementation policy have been drawn from the 
current list of indicators of efficiency of Botswana's EMIS, recent studies of the organizational 
structure and functions of Botswana's Department of Curriculum Development and Evaluation, 
and relevant literature on large-scale educational projects and indicators of educational 
efficiency. The list should be considered as tentative and preliminary, pending further study and 
further input from other practitioners and researchers. No attempt has been made to place 
indicators in order of priority of importance or worth."* 

Context 

1. Officer characteristics including age, level of education, experience and other 
background; 

2. Authority or decision-making structures including policy statements of where decisions 
are to be made and practical examples of where decisions actually occur; 

3. Sources of support for curriculum development and implementation, e.g., in-service and 
pre*service support, tran^rtation and production fiacilities, specialized expertise, private 
sector publishers' roles; 

4. Educational philosophy and aims of the educational system; 

5. Structure of the educational system as a whole, e.g. , centralized vs. decentralized system; 

6. Attitudes of personnel toward the curriculum development personnel and the roles they 
are to fulfill; 

Input 

L Financial inputs including the sub-system's annual budget as well as more detailed budget 
information such as the proportion of the curriculum department's budget going toward 
the purchase of books/workbooks and supplementary materials, development of new 
materials and supplementary materials, inservicing of new materials and in-place 
materials, evaluation of materials, and "^marketing" of new curriculum initiatives; officer 
salaries; 
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2. Sources of authority including authority to make curriculum decisions about changes in 
curriculum subjects, topics, skills, teaching methodologies, and time allocation per topic 
or subject area; 

3. Officers' current cq)abilities and preparedness for curriculum tasks including 
understanding the following processes: conducting needs assessment, developing 
aims/goals/objectives, designing/developing curriculum, evaluating new curriculum, 
developing test systems for curriculum including test specifications, test blu^rints, and 
test items, conducting summative evaluations, conducting research, managing educational 
projects, preparing and writing reports and official correspondence, writing instructional 
material, and prq)aring and conductmg workshops; 

4. The sub-system's human resource development capacity including number of training 
opportunities per year, availability of long term and short term plans for training; job 
descriptions, management approaches; training manuals, job aids, operational rules, 
regulations, and guidelines; career paths, employee turn-over; and utilization of clerical 
and other support staff; 

5. Resources including number of officers per subject area, number of officer positions 
requested during previous year, number of officer positions allocated for current year; 
amount of professional reference materials that are accessible including books and 
periodicals, numbtf of outside resource opportunities (e.g., workshops, speakers, 
conferences, exposure to systems outside of the state or country), availability of housing 
and oth^ "perks" provided by the government, number of department vehicles and 
drivers available, types and numbers of production equipment (e.g., typewriters, 
computers, printers, copiers, cameras, staplers, paper) and percent of equipment that is 
operadonai at any given time; 

6. Work environment indicators including how officers regard their jobs, their work 
environment, their preparedness to do their jobs, their effectiveness, the administration, 
and the effectivoiess of their colleagues, and how all of the above compares with the 
environments of other departments; officers' regard for career paths and the current 
criteria for promotion and further training; 

7. Contributions by outside agencies (donors, publishers, volunteer groups, communities, 
other government departments) including support in money, resources, expertise, 
transportation, equipment and training; 

8. Availability of contextual information including characteristics of students and teachers 
for whom curriculum is being designed (e.g., student reading levels or teacher abilities 
to use a variety of methodologies); school teacher and administrator pr^aredness to 
implement new programs, thdr anxieties and concerns, and their roles as educational 
leaders; student performance on examinations; types of jobs available after schooling; 



97 



ERIC 



118 



proportion of students going to further academic or vocational training or 
employment/unemployment after school; oth^ demographic information on students; 

9, Availability of planning infonnation including the sub-system's mission and long range 
and short range goals with projected dates for achieving those goals and implementation 
plans and schedules; availability of policy documents specifying roles and responsibilities 
of the units within the department; the availability of planning documents such as school 
calendars, examinaticHi schedules, school visitation schedules, publishing guidelines, 
organizational charts; 

10. Availability or utilization of communication and coordination mechanisms and structures 
to facilitate commimication and formal linkages among ministries, departments, units, 
and donor agencies as well as more informal two-way communication among the officers 
across those organizations (e.g., curriculum coordinating committee, staff meeting 
schedules, workshop schedules); 

Process 

1 . Planning functions including clarity about curriculum units' mission with the department, 
list of priorities in order of importance, existence of long and short range unit goals with 
projected dates for achieving goals, and implementation plans and schedules for achieving 
go^s; 

2. Time management including average number (per person) of sick days, personal leave 
days, days spent in meetings and workshops, days spent in work not directly related to 
producing curriculum materials within the officers' assigned subject areas; proportion of 
time spent in managing, coordinating, & communicating vs. writing, designing, & 
producing; proporticm of time per year spent conducting one or more of the curriculum 
development tasks (listed under "Input*); time taken to develop new curriculum vs. 
adapting old curriculum vs. maintaining present curriculum; number of months/years it 
takes to develop a "year's worth" of curriculum materials for the various school levels; 

3. Personnel management including the number of administrators and coordinators as well 
as support staff that are available per officer within the department; number of schools, 
students, teachers who are using curriculum products per subject area; identification of 
who controls and has the ^ropiiate information for making curriculum decisions and 
who controls the budg^ for implementing decisions; the decision-making structure within 
units; officers' perceptions regarding the degree and adequacy of consultation and 
involvement in decision-making processes; 

4. Communication including formal and informal mechanisms for internal unit coordination 
and communication among officers; intra- and interdepartmental communication 
opportunities for officers; frequency of correspondence and personal visits with school 
teachers, in service officers, and others; "routincness" of two-way communication with 
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other units and the teacher training system; degree of access to schools on an as needed 
basis; frequency of planning opportunities with other departments; timeliness of 
notification of training opportunities; 

5. Implementation of curriculum development including the numbers of officers who have 
conducted various curriculum development taslcs and processes (see ''Input'*) and the 
amoimt of time spent conducting them; 

6. Use of resources including the amount and ufiit cost of resource used and the patterns 
of use by officers; 

7. Delivery of curriculum products including number of times production or delivery of 
materials was on time or postponed, percentage of times that materials were delivered 
according to official schedule or number of days delivery was delayed; 

8. Capacity including number of development, production, delivery, and implementation 
efforts occurring simultaneously within a unit and within specific subject aieas at any 
given time (e.g., development of books/worUxx>ks/other subject materials, inservicing 
of materials, evaluation and revision of materials, and the conceptualization of new areas 
for development); 



Output 

1. Materials produced including books, supplementary materials, workbooks, teachers 
guides, and learning aids; 

2. Workshops and training exercises conducted including formative evaluation and inservice 
workshops with teachers, headmasters, inservice officers, and inspectors; 

3. Curriculum content including content analysis of materials to see if they contain what was 
intended; 

4. Utility of curriculum material and assessment including how syllabuses, materials, 
guides, and assessments are functioning and are being used; 

5. Curriculum dissemination, diffusion, and institutionalization including indicators that 
curriculum will remain functional within the school context in terms of student access to 
materials, teacher use of appropriate teaching methods, quality of teacher training, 
supervisor and administrator training, nuiteriai distribution and replacement systems, 
record keeping and reporting systems; 
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Outcome 



1 . Student achievement/attainment including school enrollments, number of students setting 
for school leaving examinations, percentage of students completing basic education cycle, 
and percentage of students achieving examination results at various levels of 
p^ormance. 

Summary 

More educational system performance research should be directed toward curriculum 
development processes and the capabilities of curriculum development sub-systems. With the 
information provided by such research, practitioners would not only be able to monitor and 
improve the efficiency of curriculum development processes being used but would also be able 
to facilitate the institutionalization of those processes better. To date, educational system 
performance research has begun considering school-level data in addition to national-level data 
in an effort to study what makes education more efficient and effective. While both levels of 
data collection are useful in providing a more comprehensive picture of educational systems 
performance, the processes that are used to develop a functional curriculum to address national 
and community educational needs seem to have been overlooked. 

This discussion has been an attempt to address the ov^sight and to suggest the types of 
information that might be useful in monitoring and improving the efficiency and effectiveness 
of curriculum development and implementation sub-systems. 
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The Impact of Sectoral Adjustment on 
the Design and Implementation of an 
Educational Management Information System: 
The Case of Guinea 



Joshua Musldn 



By the start of the 1990s, sectoral adjustment has clearly caught on as a prominent modus 
operandi of USAID and, particularly, World Bank assistance programs. Funds are released to 
governments in a sequence of tranches (or partial allocations) only after a set of pre-determined, 
jointly accepted conditims or targets are met.^ These tranche-release conditionalities regularly 
include a mix of obligations such as the adoption of policies, the implementation of studies, and 
the achievement of targets that the donor, and apparently the beneficiary, believe will promote 
fundamental positive changes in the sector recdving support and, consequently, in the broader, 
macro-economic/social welfare context. The critical aspect of the process is, then, not the 
satisfaction of the conditionality requirements— which serve primarily as monitoring or tracking 
tools— but the articulation and implementation of a coherent comprehensive strategy of reform. 
The risk is that, once the objectives of tiie program are set within a conditionality framework, 
attention may be diverted away from the implementation strategy to a more mechanical, 
narrowly focused effort to meet tiie specific conditionality targets. In this manner, the targets 
are met to the detriment of the program objectives. 



In 1990, the Government of Guinea undertook a sectoral adjustment program for its education 
sector, tile Programme d'Ajustement Seaoriel de VEducation (PASE). This program was the 
result of a process initiated by Guinea in 1984 at a national education conference. The World 
Bank, The United States Agency for International Development (USAID) and the French 
Government's Fond d'Aide et de Coopiration (FAC) have collaborated to assist tiie Government 
of Guinea in further refining and implementing a reform program to improve the quality, access 
and management of all levels of its national education system. Given the profoundly 
underdeveloped nature of Guinea's education sector— e.g., with a 28% gross national enrollment 
rate (18% in rural areas) and a per student annual expenditure on materials of just $0.20r-tiie 
foundation of the World Bank and USAID participation targeted particularly the aims of 



The contentioQ that the conditiont are artkulatad and agreed upon through a negotiated process involving two 
co-eq[aal partners is defended strongly by the World Bank and USAID. This belief is refuted with similar 
conviction by their African government counterparts. These officials perceive the power of the Bank as lender 
or USAID as grantor to represent an inordinate amount of influence in setting the conditions for their countries* 
sectoral adjustment programs. They acquiesce to the conditions in order to receive much-needed loans and 
grants* 
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increasing the flow of resources to the school and assuring their rational utilization. More 
specifically, these ccmcems included facilitating and accounting for the flow of resources 
(material, human and financial) from the central authorities to regional, or prefectural, education 
offices and, ultimately, to individual schools. The French are participating in the PASE 
primarily in the area of teacher training. 

Several conditionalities have been accepted by the Government of Guinea in negotiations with 
the donors, reflecting a joint conmiitment to promoting and executing Guinea's national 
education reform agenda. The targets idratified in the conditionalities have been formulated to 
help guide and monitor progress in the different areas of this program. These conditionalities 
must be met before any of the donors approve subsequrat dollar tranche releases. 

US AID has contracted Florida State University (FSU) to provide technical assistance in the areas 
of finance and administration to support the Ministry of Education in establishing management 
information collection, analysis and reporting systems. The identiiication and design of these 
complementary, often overlapping, systems is intended to support the development of a technical 
capacity within the Ministry of National Education to administer the operations and finances of 
the sector. A schedule has been agreed upon by the Government and USAID to evaluate the 
program, a ""conditionalities review," at three times over the life of the present program: (The 
same schedule has been adopted by the other donors, although each has set its own dates for this 
with the Government.) With each successive evaluation, a new set of targets will be assessed. 
If met, these are intended to rq>resent further advancement by the PASE and the Government 
towards meeting the overall reform objectives. 

During the first phase of the program, the overall goal of the USAID-Guinea conditionalities is 
to assure that political and structural frameworks exist for the program's implementation. 
Additionally, plans are required of the Government of Guinea for meeting basic long-term 
objectives, including the presentation of a set of baseline data against which to assess progress. 



The second phase evaluation focuses more precisely on actual changes in Ministry policies and 
practices and on the Government's real commitment (in dollars and personnel) to the education 
sector. This is represented in targets such as: (1) an increase in the proportion of total 
Government expenditures allocated to the education sector; (2) evidence of an increased portion 
of Ministry of Education expenditures going to recurrent budget items, from 14 to 17 percent; 
and (3) the existence of a plan for achieving the equitable participation of girls and jral students 
in schools nationwide. 

The third and final release of USAID funds to the Government in support of the PASE reform 
efforts is tied mainly to evidence of further progress in the same areas addressed in the second 
evaluation. By this time, the proportion of recurrent Government expenditures on education 
must have reached a target of 21 percent. Further evidence must also be provided of reasonable 
progress toward the targets specified in the plans for gender and regional equity, evident in 
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school constructicm and rdiabilitation budgets, teacher assignments, salary and non^salary annual 
recurrent expenditures, and girls' and regional enrollment rates* 

As stated above, the different conditionalities have been selected to indicate specific, more 
profound advances in the national education reform process* This is demonstrated generally in 
two ways. First, a positive evaluation should show that the objectives of the reform effort are 
being met; e.g., that the distribution and management of human and material resources are 
occurring as planned, generating the anticipated improvements in school quality and equitable 
access. Seccmd, a successful evaluation is understood to serve as evidence of the Ministry of 
Education's improved capacity as an institution and of its staffs heightened ability to administer 
and implement Guinea's schools. 

The conditionalities agreed upon between USAID and the Government of Guinea reflect those 
agreed upon between the Government and the World Bank, whose involvement in the education 
reform program preceded that of USAID. (The different donors, including the French, do 
attempt to assure consistency in their different programs, such as with the occurrence of a 
weeldy donors/PASE meeting, but collaboration and agreement is rarely as easy as 
conversation.) The first priority for the Bank's program was to receive a statement by the 
Govemment committing itself to a comprehen^ve outline for the reform of the country's national 
education policy and practice. Various committees, comprised of members who are botii 
directty and indirectiy affected by the sectoral adjustmrat program, have been formed to 
constitute a political, administrative framework for the PASE. Also included at tiiis initial stage 
are (i) an internal reorganization of the Ministry of National Education/State Secretariate for Pre- 
University Education (MEN/SEEPU)* and (ii) tfie establishment and operation of regular 
monitoring procedures within the MEN/SEEPU for both budget allocations and actual 
expenditures in the education and training sector. Additional conditionalities involve a 
combination of quantitative targets— for education staff redeployment, proportional Govemment 
budget allocations to and within the sector, and efficiency measures— and more specific structural 
strategies—for introducing a computerized personnel management system and for preparing and 
executing rationalization plans for post-secondary and technical education. 

Both the World Bank's and USAID's conditionalities attempt to achieve two major objectives: 
(1) to establish an equitable, efficient allocation of resources among Guinea's education sub- 
sectors and target populations; and (2) to assure tiie existence of valid, consistent, -transparent- 
accounting and reporting procedures that will reveal the level of success in meeting objective 
one. The related targets, strategies and structures are articulated in a manner that reflects a 
mutual assumpticHi that the mechanisms and resources brought to bear to satisfy the specific 
conditionalities will yield positive permanent effects that will endure well beyond tiie life of tiie 
program. In particular, program success depends on the fulfillment of a few expectations, or 
hopes: 



SEEPU WM named an independent Minittry as of February, 1992. 
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i. Expenditure patterns onployed by the GoTemment and the l^finistry of 
Education established during the life of the program will carry over to the 
post-program period (an explicit assumption). The level and proportional 
allocati(Hi of expenditures agreed upon between the Government of Guinea and the 
World Banlc and USAID, and (a crucial point) permitted by the temporary untied 
funding paid straight into the Treasury by these two donors, are expected to be 
sustained after the external financial support ends. The donors justify this 
assumpticm in a few ways. For one, they assert that adequate finances will exist 
ik the country's coffers after 1996. The Bank's projections of the Guinean 
economy indicate that national growth in exports and production will compensate 
for any shortfiadls resulting from die end of outside funding. Further confidence 
derives from die proposal diat once the new ^pending patterns are established 
within the Government system, they will be largely intractable, due to a 
combination of institutional habit and the collective expectations of the various 
affected client parties— students, parents, teach^, prefectures,... 

ii. Expenditure targets and other measures reflected by the conditionalities will 
yield improvements in the quality of the education provided by the system (an 
implicit and explicit assumption). This proposition appears to be valid insofar as 
the program results in inoreased access to schooling for presentiy excluded groups, 
with new school construction and rdiabilitation programs, staff redeployment and 
revised spending plans. Program provisions mandate that the proportion of the 
total recurrent costs allocated to prinuuy education in the second year of the PASE 
be at least 34 percrat of the total national education budget. (This category has 
already reachwi 35 percent.) The proportion of non-salary recurrent spending 
should reach at least 17 percent by die end of year two, and 21 percent by the 
program's end. (The 21 percent target has also already been achieved.) Per 
primary student non-salary annual expenditures are mandated to increase from 
about $0.20 to about $4.00 in the first year and maintained over the reform's life. 
As the PASE implements systems that assure that more money go to pedagogical 
purposes and as this money is distributed more equitably throughout the country, 
heightened quality is anticipated. This is particularly expected as tiiesc 
expenditures arc associated with proposed improvements in pedagogical programs 
introduced with the aid of FAC— providing teacher training programs — , and die 
World Bank— promoting a multi-grade classroom strategy, especially in rural areas 
to increase enrollments. 

iii. Hie ability of the Ministry of Education to report on conditionalities will 
indicate an increased capacity for carrying out appropriate^ established 
monitoring procedures and the existence of viable systems within the Ministry 
for operating the national education system outside of the PASE context (an 
implicit assumption). The reporting requirements associated widi World Bank and 
USAID conditionalities involve the performance of numy functions— e.g,, line item 
budgeting, presentaticm of receipts and other spending documentation by local 
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prefectural education offices, and national accounting reports that the Ministry 
is presendy poorly equipped to undertake.^ Neither tiie necessary professional 
cs^acity nor zppropmic systems or procedures for doing this exist With the 
institution of new administrative structures and procedures and with the practice of 
repotting comprehensive education data within the donor review framework, it is 
anticipated, finally, that the Nfinistry of Education will have embraced and 
mastered both the idea and the practice of data-based management and decision- 
making by program's end in 1996. 

Some of these assumptions permit greater opportunity for confidence than do others, but none 
is guaranteed. Rather, each must be analyzed carefully, asking if the reporting systems and the 
actual quantitative and institutional targets articulated in the PASE conditionalities will truly 
effect lasting reforms in school quality and in accounting for and reporting cash and materials 
flows within the education sector. A key consideration in this determination is the direct 
intention of the design and implem^tation of the prescribed financial and administration 
systems, policies and targets. For example, the accelerated achievement of spending percentage 
targets indicated above is attributable primarily to increased tertiary level expenditures; yet the 
purpose of these targets was to raise tiie level of primary school inputs. 

It must be asked, are the strategies and policies chosen primarily to meet short-range 
conditionality indicators or has there been a purposeful, thoughtful effort to effect lasting 
improvements in the efficiency and quality of the administration of the national education 
system? More simply, it can be asked if the individual assumptions have a legitimate chance 
of becoming reality, regardless of the design and execution of the program. This concern is 
particularly meaningful as it relates to what can only truthfully be considered a wishful 
assumption (#1) that donor-supported expenditure levels within the Ministry will persist after the 
end of the program. 

The priority given by the World Bank in its program to the creation of committees and an 
internal reorganization of the MEN and MEPU administration seems obviously formulated to 
address more long-term, capacity-building aims of the PASE. The use of conditionalities 
represents, in this instance, a more far-reaching joint initiative (involving US AID and the FAC) 
to provide technical assistance that should help galvanize the Guineans ability to administer their 
education system. Satisfying the conditionalities is intended to be basically a secondary, or even 
somewhat symbolic goal, one that is demonstrative of the more fundamental objectives of the 
sectoral reform process. Were it different, simply a matter of satisfying the targets enumerated 
in the respective USAID and World Bank conditionalities, the permanent impact of the PASE 
would be much more doubtful. In this manner, more confidence can be afforded in considering 
assumption number 3. 



Given ih% hex that thtf years ago, the lector did not even operate with a budget, the critical need for this 
support is very understandable; thus USAID*s emphasis in this area. 
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Looking specifically at the USAID ccmtribution, the primary concerns are (1) to identify the 
prominent technical needs of the MEN/SEEPU in tfie areas of finance and administration and 
(2) to design and implement a comprehensive stiat^y to address these. In this manner, 
satisfaction of the conditionalities are perceived as a omsequence of addressing the long-term 
cs^acity and systemic requirements of the national education system, not as the ultimate goal. 
Clearly, when progress review periods arrive, the associated Ministry and Project staff pay 
special attention to prepare, ddiver and defend the required documentation demonstrating 
satisfaction of the various conditionalities. But the plan is that it will be increasingly possible 
to rely upon existing, competent systems and personnel for the routine preparation of these and 
other similar rqwrts. (This has not been the case for either of the first two conditionality 
reporting exercises.) 

The proposed configuration of the FSU/US AID technical assistance program includes two long- 
term technical advisors and about 30 person-months of short-term consultants to be used over 
the two-year Ufe of the FSU/USAID Project (USAID is presently committed to a full five-year 
program) in various areas of education finance and administration. The position of Principal 
Technical Advisor (FT A) has been designated to sit within the Technical Secretariat of the 
PASE, collaborating with the PASE National Coordinator and the FAC Technical Assistant for 
Adjustment to articulate and assure the proper implementation of systems and policies' for an 
effective, comprdiensive Guinean education reform. 

Working in the areas of finance and administration, FSUAJSAID is concerned most prominently 
with the work of the DAAFs and S AAFs (Directions and Services des Affaires Admimstratives 
et Fifumciires). The two DAAFs serve as the central authority overseeing the MEN's and 
MEPU's entire finandal and administrative affairs and are located in Conakry. These offices 
are the most directly affected by the PASE reforms and, consequently, their work relates most 
closely to the conditionalities. The SAAFs serve essentially as prefectural satellite units of the 
DAAFs. Their major functicms are to serve as the link between the DAAFs and the schools, 
managing and monitoring the distribution of resources to the schools and reporting on these 
activities back to the DAAFs. The specific responsibilities and functions residing within the 
DAAFs are essentially: 

• to prqpare the annual education budget for submission to the Ministry of Finance; 

• to manage the expenditure of the budget, assuring that the monies are assigned and 
disbursed properly, according to plan; 

• to assign and manage the administrative and teaching personnel of the national 
education system; 

• to manage the equipment, materials and properties of the MEN and SEEPU, 
including the implementation of renovation and new construction activities; and 
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• to account for the various resources (financial, material and human) of the national 
education system. 

The FSUAJSAID program of technical assistance to the PASE covers all aspects of this list. 
In addition to the two residrat technical advisors (one of whom has been in place since 
September, 1991, the other arriving in June, 1992), a relatively lengthy list of short-term 
expertise is planned for the areas of budgeting, procurement and tendering, properties control, 
mat^ials management, management information systems, computerization, managemmttraining^ 
and internal audit. Two areas that do not M directly within the rubric of finance and 
administration but found within the USAID areas of responsibility and covered by the 
conditionalities are also covered by the short*term consultant plan: monitoring and evaluation, 
and gend^ and regional equity in school access. The PASE may identify other needs for related 
technical expertise as the program develops. 

The Project plan is to inject the different technical experts into the overall program at strategic 
times to build up the overall capacity of the MEN/SEEPU to manage and administer Guinea's 
national education system. FSU/USAID has identified a strategy (continuously being refined 
with fiirtha: reflection and evolving contextual developments) that assigns to the series of 
technical experts three basic tasks: 

1. analyze current systems, levels of competence and needs associated with the 
particular areas of expertise operating within the DAAFs and SAAFs; 

2. identify strategies, systems and a training agenda, and prepare appropriate 
documentation, with the aim of upgrading the individual and systemic capabilities 
of the MEN/SEEPU; and 

3. initiate the implementation of the articulated measures, procedures and policies, 
undertaking regular monitoring and evaluation e^orts. 

As suggested above, it is possible to report the status of the conditionality indicators with a 
functional, or "punctual," intervention without engaging in this manner of capacity-building 
strategy. A technical expert could easily collaborate with his or her highly placed Guinean 
counterpart(s) to gather any necessary documents and prepare various reports alone, with 
minimal input and participation from the associated Guinean staff who will evmtually be charged 
with the routine preparaticm of such data and documents. Similarly, the various committees and 
new institutional structures included among the Bank's conditionalities could be created that 
senre little more than symbolic purposes. Budget levels could be raised, teachers could be 
redeployed, and schools could be built and renovated, all achieved and repomd satisfactorily 
without instigating if any real fundamental improvements in either the quality and coverage of 
the education system or the ability of the Ministry of Education to administer the sector have 
occurred. 
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The 1992 annual budget exercise of the MEN/SEEPU exemplifies the first of these caveats. The 
elaboration of this budget for submission to the Ministry of Finance coincided directly with the 
arrival of two FSUAJSAID technical assistants. The budget's preparation involved essentially 
a scraml>le by five people—the two Guinean Directors of the DAAFs and three technical 
advisors (the two FSUAJSAID consultants and one from the World Bank)— to gather all the 
necessaiy documentation and prq>are an official report. This process only resembled vaguely 
a formal budget preparatiim process, one involving systematized analysis and submission of 
documentaticm from the many different levels of the education system. There surely remains 
much progress to be made before the Guineans are able to take over this process indespendently 
as a routine task. This is particulaiiy true of the collection and submission of documentation by 
schools and administrators at the prefectural level. 

The tranche review process provides perhaps an even more acute example of the dangerous 
potential to satisfy sectoral adjustment conditionalities without effecting an enduring impact upon 
the ability of the targeted institution to satisfy routine finance and administration duties. As 
argued above, if it were just a matter of meeting this discrete set of targets, this could be 
achieved with littte impact upon national technical or institutional ability. An expatriate team 
could effectively guide the PASE process in an ess«itially mechanical way— assuming an 
agreeable or pliant political will— to prepare the necessary reports, organize the prescribed 
committees, and plan and execute the various reforms (e.g., expenditure allocations, 
redqployment, school construction) delineated in the conditionalities. The review process would 
be satisfied and the subsequent tranche could be released. The ultimate goal is, however, that 
this information will be available systematically from a variety of different ministerial DAAF 
units engaging many different Guinean staff and administrators. The World Bank's adherence 
to this s^roach is reflected in their insistence upon a postponement of their tranche review 
mission ftom its original November dates to F*ruary, when the Government of Guinea 
indicated that it would be ready for the evaluation. 

In a certain way, the demands placed upon the system by the conditionalities reporting process 
can be seen to obstruct the ciqwcity-building and systems development aspects of the PASE. 
Quite practically, the level of effort and time required to prepare three differrat sets of 
conditionality documents (one each for the World Bank, US AID and FAC) leaves relatively little 
time for a planned program of technical support to the DAAFs and SAAFs. Taking this year 
as an example, starting in early September, the DAAFs and the PASE (including the expatriate 
assistants), were engaged fully in trying simultaneously to prepare the Ministry's annual budget 
submission and the conditionalities report for the World Bank's second tranche review, 
scheduled for early November. Due to the Ministry's request for an extension, the review was 
put off until February, while the DAAFs and the PASE continued to expend much, if not most, 
of their effort to gather and prepare the necessary information. The result of the Bank's review 
was a conditional s^roval, contingent upon the presentation of more documentation. This 
process took about another month and a half, overlapping with efforts to prepare the 
documentation for USAID's upcoming tranche review in June. This is apparently a less 
involved process, but a similar evaluation conducted recently in Namibia resulted in an 
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additional 60 days of work to the Ministry officials. And then there is the French review, which 
should be much more relaxed. 

Expatriate resident experts, donor officials, and Ministry administrators and managers are 
required to operate in a crisis mode to prepare documents for the discrete targets of the 
particular tranche review. Consequently, this also detracts from the Ministries' execution of the 
routine reporting procedures placed upon them by the Government. First, the tranche review 
process skews the information the Ministries gather to ends defined by the sectoral adjustment 
program, rather than to comprehensive opoational needs (although the two are meant to be 
compatible). Second, as i^emplified above, the tranche review process simply absorbs an 
exorbitant number of Ministry and technical assistance person-hours to permit a rational schedule 
of joint capacity-building and routine management and reporting. This is hardly a mode of 
operation which the donors would choose purposefully to transfer to MEN and MEPU manage 
and technical staff. 

Li both the World Bank's and USABD's programs, as for other donors, there is a fundamental 
risk that excessive attention wiU be paid to the conditionalities, to the detriment of the objectives 
which these are intended to represent. In this situation, satisfactory tranche review outcomes 
might be inaccurately interpreted as an improved ministerial ability to implement an enduring 
reform program. This possibility should not be underestimated. 

The FSUAJSABD program takes precautions against this risk by not accepting an exclusively 
report-based approach to its Project. The principal focus, as stated above, is to create an 
institutional czpadty for reporting, along with a mastery of all of the associated preliminary and 
subsequent managerial and finandal functions: a comprehensive management information system 
(MIS). In addition to the design and development of a series of financial and administrative 
systems and procedures for the areas identified above, the capacity-building process will involve 
the following activities, among othtts: 

• zn ad hoc and, eventually, an institutionalized consultative process involving 
affected Guinean administrators and technicians to inform the MIS design, 
development and evaluation process and to contribute to each of the three 
subsequent activities; 

• the development of a series of operational manuals, guidelines and policies for each 
of the DAAF and SAAF functions; 

• the intrtxluction of new technology (equipment and systems) selected to facilitate 
the work of the DAAFs and SAAFs, increasing the accuracy and efficiency of 
these offices; and 

• the design and implementation of a scries of training activities, including individual 
sessions or extended programs of topic-specific training, on-going apprentice-type 
transfer of skills, and counterpart collaboration with outside technical experts, to 
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enable the various staff to master and take advantage of the new, improved 
software and hardware. 

At this early stage of the assistance program, it appears that the strategies and support 
undertafcen by the Bank and USAID to meet conditioiiEdity objectives are successful targeting 
more long-term, cj^ity-building, structural issues. As has been argued, this situation is not, 
however, a necessary outeome of pursuing and reporting the conditionalities. Rather, it must 
result from a purposeful effort by the donors to establish bom fide finance and administration 
systems that will assure the manner of reporting required of a fully operational national 
education system and only secondarily for the tranche review process. The Guinea case shows 
that EMIS planners must consider very strongly not only a system's information requirements 
but also the contextual framework in which the system will operate, sectoral adjustment 
objectives notwithstanding, to create a permanent improved management capacity that is able, 
if indeed truly willing, to implement and maintain a meaningful, lasting reform. 
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